Joey git-annex devblog
day 410 better JSON for metadata

I've had to change the output of git annex metadata --json. The old output looked like this:


That was not good, because it didn't separate the metadata fields from the rest of the JSON object. What if a metadata field is named "note" or "success"? It would collide with the other "note" and "success" in the JSON.

So, changed this to a new format, which moves the metadata fields into a "fields" object:


I don't like breaking backwards compatability of JSON output, but in this case I could see no real alternative. I don't know if anyone is using metadata --batch anyway. If you are and this will cause a problem, get in touch.

While making that change, I also improved the JSON output layer, so it can use Aeson. Update: And switched everything over to using Aeson, so git-annex no longer depends on two different JSON libraries.

This let me use Aeson to generate the "fields" object for metadata --json. And it was also easy enough to use Aeson to parse the output of that command (and some simplified forms of it).

So, I've laid the groundwork for git annex metadata --batch today.

Anna (Anna and Mark: Waldeneffect)
Keeping the fat on our goats

I was a bit dubious of our vet's diagnosis that Aurora was merely suffering from internal parasites. After all, would she be fine one day and then nearly comatose the next in that case? But now I'm thinking he was right. Because ever since being flushed out with a vast array of pharmaceuticals, our doeling has been growing faster and plumper every day.

Walking goats

Her mama, on the other hand, is starting to drop below the perfect 3.0 body-condition score. That's perfectly normal with heavy milkers, but I'm still going to try to plump her up with some extracurricular grazing in hopes we can keep her fat enough to Full belly clubmilk through the winter.

It's a long shot for a first freshener to milk through, but it sure would be nice not to have to worry about the hassle of breeding and kidding this fall and next spring. August and early September will be the deciding time because I'd like to breed around Halloween if we're going to have to dry Artemesia off and give her time to recover before turning her back into a milk jug once again.

Anna (Anna and Mark: Waldeneffect)
Summer mushrooms
Summer collage

Despite the current bone-dry conditions, mushrooms are popping up all over the woods.


There's a certain zen wonder in not being able to identify most of them but still enjoying their beautiful shapes and colors.

Goat eating fallen leaves

Meanwhile, the goats are much more interested in the first golden tulip-tree leaves filtering to the ground. To each her own!

Joey chatter
seems familiar

When working on large scale projects, especially potentially isolating ones, maintaining perspective can be difficult. Small problems and pressures can feel amplified. The natural ebb and flow of optimism and pessimism can easily cloud any sense of progress.

-- Cheese on porting Day of the Tentacle to Linux

skt-ko (Tomoko)



We went to Hi-Wire brew room celebration!

Anna (Anna and Mark: Waldeneffect)
Harvesting onions early
Onion harvest

I usually wait to harvest our onions until the leaves have dried down. But the summer abruptly turned wet and I began noticing a few signs of rot. Time to get our precious bulbs out of the ground before they go bad!

I harvested perhaps three-quarters of the onions Tuesday, filling up three bushel baskets and then transferring the contents over onto one of our curing racks. Maybe I actually planted too many this year?

Joey chatter
stones in shoes

Afternoon creek walk the same way we went seven years ago. The most fun you can have with a stone in your shoe for two hours.

Also, my sister found her lost 5 gallon bucket, tangled in a downed tree a mile from her ford. And took a swim with it!

Joey chatter

Question is, would you feel comfortable using such a thing?

Anna (Anna and Mark: Waldeneffect)
Historic Fairview, Abingdon, Virginia
Old-fashioned thresher

Belts and gearsKayla and I took in the antique tractor show at Fairview in Abingdon Friday. It was another perfect girl's day out, with an extremely well-behaved baby, fascinating old implements (like this thresher --- look at all those belts!), and quite a bit of historical education as a bonus.

Growing flax

OuthouseThe interpretive signs were top-notch, full of information I'd never considered. For example, did you know that the crescent moon traditionally shown on an outhouse was meant to designate the lady's room? Way back when the average American couldn't read, stars were for guys and moons for girls...but men's outhouses tended to get run down and didn't last. Thus the crescent-moon-marked outhouses dominating the colonial landscape.

And speaking of outhouse traditions, hollyhocks were usually planted around the outhouse as a way to draw the attention without forcing a lady to request directions to the necessary. Lamb's ears with their silvery ears did the same job at night (while also providing backup toilet paper).

Oh, and the other picture in this section is flax. I was proud of myself for guessing its identity...and Kayla was proud of herself for capturing the plant matter before it made it into baby D.'s gaping mouth.

Organic dyes

Plant dyesIn the air-conditioned comfort of the interpretive log house, we took in several beautiful displays pertaining to history and crafts. On the history front, I was intrigued to learn that I-81 (the big highway that runs through our region) began as a buffalo trail but soon became a major thoroughfare between Philadelphia and the Frontier.

In the late eighteenth and early nineteenth centuries, the so-called Great Philadelphia Road was used for wagon trains of settlers (many from Europe) moving to new land and also for people from our region driving their sheep, pigs, and cattle to market in the City of Brotherly Love. Can  you imagine making that 500-mile trek at the tail end of a herd of swine?

Percheron horse

Pig overallsThe upshot? Best girl's day out yet! Think we can top this in August, Kayla?

Anna (Anna and Mark: Waldeneffect)
Soil Amendments for the Organic Garden

Soil Amendments for the Organic GardenI'm excited to announce that the final book in my Ultimate Guide to Soil series is now available for sale!

This one is long and in-depth but very heavy on the pictures so hopefully still easy to read. In addition to rundowns on more conventional garden amendments, I've summed up our experiences with bokashi, black soldier flies, humanure, biochar, and much more.

Here are the buy links in case you want to give it a try:

What early reviewers had to say:

"Thanks to her Ultimate Guide to Soil series, I am enjoying robust plant growth and harvesting gorgeous vegetables and herbs from my garden." --- M

"I know more about composting and soil amendments now than I did before reading this - and I've been composting for 30 years, so I thought I knew it all!" --- Colin B. McGee

Thank you in advance for any support, from buying a copy to telling a friend to leaving a review. You are why I write.

Anna (Anna and Mark: Waldeneffect)
Tether training
Goats grazing hedge

Goat pulling against its leashNow that Punkin has moved on to his new home, I'm finally able to get back to tethering our herd in the morning while I garden. Artemesia was thrilled at the opportunity, but our little doeling wasn't so sure about the idea of collars and leashes.

She had a small fit, pouted for a while, then decided mimicking her mother was probably a better way to utilize her time. I figure by the end of the week, Aurora will be just as good at being tethered as Artemesia is.

Joey git-annex devblog
day 409 --branch

A common complaint is that git annex fsck in a bare repository complains about missing content of deleted files. That's because in a bare repository, git-annex operates on all versions of all files. Today I added a --branch option, so if you only want to check say, the master branch, you can: git annex fsck --branch master

The new option has other uses too. Want to get all the files in the v1.0 tag? git annex get --branch v1.0

It might be worth revisiting the implicit --all behavior for bare repositories. It could instead default to --branch HEAD or something like that. But I'd only want to change that if there was a strong consensus in favor.

Over 3/4th of the time spent implementing --branch was spent in adjusting the output of commands, to show "branch:file" is being operated on. How annoying.

Anna (Anna and Mark: Waldeneffect)
Ice-cream maker

Ice cream makerThis summer has been averaging 1 to 3 degrees hotter than previous summers on our farm. That doesn't sound like much...until you realize that we spend a lot of time working outside and I'm still stubbornly avoiding air conditioning. (Mark has AC in his man cave.)

Long term, we're pondering all kinds of passive-cooling systems. Short term, the solution was obvious --- an ice-cream maker!

This little machine makes a great long as you put in a little TLC during the freezing process. Unfortunately, the procedure isn't set-it-and-forget-it. Instead, it's necessary to run a firm spatula around the inside every three to four minutes during the course of the freeze cycle.

Without that step, you end up with ice cream stuck so hard to the nonstick surface that you can't pry it loose and end up having to slowly but surely scoop it into your mouth as it thaws. Oh, the hardship! No, Mark, that's not what I'm doing while you're cooling off in your man cave. Really....

Churning ice cream

Ahem, anyway. In case you're interested in following suit, I'm still using a slightly modified version of this recipe. Mark got sick of mint and I realized I could downgrade the chocolate chips a bit (although not much or the consistency goes off). The final recipe for use in a two-quart ice-cream maker includes:

  • 6 cups rich goat milk (divided)
  • 1 cup plus 2 tablespoons of cocoa
  • 1 cup plus 2 tablespoons of sugar
  • 1/2 teaspoon salt
  • 1 cup of dark chocolate chips
  • 6 tablespoons of cornstarch
  • 1 teaspoon of vanilla extract
Soft-serve ice cream

See the previously linked post for cooking instructions and be sure to cool overnight before tossing in the ice-cream maker. The end result is soft-serve consisetency, so Mark and I like to ladle the contents into eight individual-serving containers and put them back in the freezer for 12 hours before eating. Enjoy!

Joey chatter

Unusually relevant places I have been: Slovenia

Joey git-annex devblog
day 408 release day

First release in over a month. Before making this release, a few last minute fixes, including a partial workaround for the problem that Sqlite databases don't work on Lustre filesystems.

Backlog is now down to 140 messages, and only 3 of those are from this month. Still higher than I like.

mark (Anna and Mark: Waldeneffect)
Goodbye Punkin
Punkin meets his new owner.

Punkin is now with his new herd where he'll be the new stud goat.

It was a little sad to see him go, but in the end we're glad he found such a good home and a new purpose in life.

Re: Debugging over email

Lars wrote about the remote debugging problem.

I write free software and I have some users. My primary support channels are over email and IRC, which means I do not have direct access to the system where my software runs. When one of my users has a problem, we go through one or more cycles of them reporting what they see and me asking them for more information, or asking them to try this thing or that thing and report results. This can be quite frustrating.

I want, nay, need to improve this.

This is also something I've thought about on and off, that affects me most every day.

I've found that building the test suite into the program, such that users can run it at any time, is a great way to smoke out problems. If a user thinks they have problem A but the test suite explodes, or also turns up problems B C D, then I have much more than the user's problem report to go on. git annex test is a good example of this.

Asking users to provide a recipe to reproduce the bug is very helpful; I do it in the git-annex bug report template, and while not all users do, and users often provide a reproducion recipe that doesn't quite work, it's great in triage to be able to try a set of steps without thinking much and see if you can reproduce the bug. So I tend to look at such bug reports first, and solve them more quickly, which tends towards a virtuous cycle.

I've noticed that reams of debugging output, logs, test suite failures, etc can be useful once I'm well into tracking a problem down. But during triage, they make it harder to understand what the problem actually is. Information overload. Being able to reproduce the problem myself is far more valuable than this stuff.

I've noticed that once I am in a position to run some commands in the environment that has the problem, it seems to be much easier to solve it than when I'm trying to get the user to debug it remotely. This must be partly psychological?

Partly, I think that the feeling of being at a remove from the system, makes it harder to think of what to do. And then there are the times where the user pastes some output of running some commands and I mentally skip right over an important part of it. Because I didn't think to run one of the commands myself.

I wonder if it would be helpful to have a kind of ssh equivilant, where all commands get vetted by the remote user before being run on their system. (And the user can also see command output before it gets sent back, to NACK sending of personal information.) So, it looks and feels a lot like you're in a mosh session to the user's computer (which need not have a public IP or have an open ssh port at all), although one with a lot of lag and where rm -rf / doesn't go through.

Anna (Anna and Mark: Waldeneffect)
Basket case
Basket cat

Question: Am I the only one who turns into a bit of a basket case at the height of the summer gardening and preserving season?

Answer: No, Huckleberry's quite willing to join in the fun!

Joey chatter

is an anagram for

Anna (Anna and Mark: Waldeneffect)
Changing of the caprine guard
Sleek goats

Alert goatsThere's a new darling doeling in town....

Back in the winter when I named Aurora before she hatched, I asked Mark if he thought our hypothetical doeling would be as sweet as her mother. "Sweeter," he said promptly. "Impossible!" I replied.

But motherhood is tough on everyone, and I'm forced to admit the transition to a new life stage has taken a tiny bit of shine off Artemesia's halo. She's still the world's greatest milk goat. She lets me milk her without locking her head in the stanchion, goes wherever I ask without a leash, and generally acts like the perfect herd animal.

Goat familyOn the other hand, making 6 cups of milk per day takes a lot of calories. So Artemesia doesn't have as much time to caper as she used to. Instead, she generally needs to hunker down and eat.

Enter doeling 2.0....

At three months old, Aurora is just a little younger than Artemesia was when we brought her home. And the sweetness of a teenibopper goat is very much in evidence. Sunday, our darling doeling snuggled up to me as I read in the woods and let me stroke her soft fur for quite a long time. And I realized --- even though this sounds like heresy --- that Mark might be right. Aurora might be even sweeter than her mother.

Joey git-annex devblog
day 407 lazy sunday

Noticed that in one of my git-annex repositories, git-annex was spending a full second at startup checking all the git-annex branches from remotes to see if they contained changes that needed to be merged in. So, I added a cache of recently merged branches to avoid that. I remember considering this optimisation years ago; don't know why I didn't do it then. Not every day that I can speed up git-annex so much!

Also, made git annex log --all show location log changes for all keys. This was tricky to get right and fast.

Anna (Anna and Mark: Waldeneffect)
Making like ants
Crazy butternut patch

I went out to try to take a photo of our butternut patch with a human in it for scale, but ended up getting this odd selfie instead. So I've inset another photo of most of the patch to give you an idea of the impressive coverage of those squash vines just a couple of months after planting.

Red tomatoes

Storage onionElsewhere in the garden, it's starting to turn into preserving season. We're putting away our usual soups as well as lots of frozen green beans and okra since the latter turned out to be a crowd pleasure this winter. It looks like we'll be harvesting our onions this week too. It's definitely time to make like ants if we want to eat well during the cold season ahead!

Anna (Anna and Mark: Waldeneffect)
Moving the last garden bed

Moving garden bedsI'm just about done with my quest to consolidate all of the best soil into the sunniest part of the garden.

On the plus side, my gardening workload has decreased because I get more produce from a smaller space.

On the minus side, I haven't had room to plant a single bed of buckwheat all summer. (Fall cover crops will still go in as planned.)

Now the big question --- what to do with the shady spaces left behind? Next time dry floodplains and working four-wheelers align, I'm hoping to drag in another round of livestock panels to build more goat pastures.

Because having Artemesia standing inches away from my sweet corn watching me weed is a good idea...right?

Anna (Anna and Mark: Waldeneffect)
Harvesting honey from a Warre hive
Honey jars

Hive conversion successAs Mark mentioned yesterday, we finally harvested honey for the first time in six years. The long-time reader might assume that means my conversion back to Langstroth hives while keeping my chemical-free strain of bees going was just what the doctor ordered. And you wouldn't necessarily be wrong...but you wouldn't necessarily be right either.

The jury's still out on which hive system makes more honey because I harvested about the same amount from both the primarily Langstroth hive and from the Warre hive. In both cases, the honey overload is due to an excellent nectar season that culminated in our basswood blooming fully for the first time in nearly a decade. Lots of nectar makes lots of honey, regardless of the type of hive.

Bee brood

So how much honey did I take? It's complicated. Initially, I planned to harvest the one remaining Warre box on top of our converted hive since I'd found lots of brood and eggs lower down on Langstroth frames and knew the queen had vacated the attic. But once I got that box home and started pulling out the frames, I discovered that there was still brood at the bottom. So I harvested three full frames of honey and went to put the Warre box back with its brood intact.

But I really wanted to call that conversion complete, so I put the box back on the other hive instead of the one I originally took it from. And while I was in there, I discovered that the Warre hive had a very full box of capped honey as well as at least two other quite heavy boxes. So I snagged the former rather than towering their living quarters over my head.

Low-tech honey extraction

To cut a long story short, I think I ended up harvesting about 1.3 Warre boxes in the end. Rather than going high-tech, I simply squeezed each frame into a strainer and ended up with about 2.5 gallons of honey. A very sweet reward after years of trial and error!

Anna (Anna and Mark: Waldeneffect)
Wholly holy, holey cardboard
Holey cardboard

After years of kill mulching, I've become a bit of a cardboard connoisseur. Corrugated vastly trumps uncorrugated. Smaller flaps mean more ground coverage than larger flaps. And produce boxes with holes in the middle? I used to think those were a pain in the butt...until now.

It occurred to me as I prepared to plant some summer squash in the site of last year's compost pile that weeds were going to pop up like mad in the newly bared ground. So I laid down a kill mulch first, using a holey box as my central row. Then I poked a single squash seed into the soil beneath each hole.

A month later, the squash are thriving and I've finally decided I love those holey produce boxes. What part of the waste stream will I find a perfect use for next, I wonder?

Joey chatter

First 10 minutes using my PocketCHIP: apt-get install nethack-console and handed it to my sister who has never nethacked before.

Anna (Anna and Mark: Waldeneffect)
Weaning a goat
Giving a goat a shot

Punkin got his followup CD&T injection this week, which means he's now ready to become the stud at a nearby friend's farm. We're Goat feederjust waiting on that friend to recover from eye surgery so he can take on the handful that is our 11.5-week old buckling.

(No, Punkin, the mineral feeders were not intended to be eaten out of in that manner.)

Punkin is the first goat I've ever weaned, and after watching him I'd say that eight weeks old probably is too young to go off milk for most goats. Our buckling came out of his weaning at a perfect, svelte 3.0 on the body-condition chart...but that's only because he began with rolls of fat around his shoulders and because I let him nurse at slowly lengthening intervals for the first two weeks. A skinnier goat or the recipient of a cold-turkey weaning would have had a much harder time making the transition to grass.


In the meantime, Punkin has gotten used to being in a neighboring pasture rather than being a true member of the herd, and he seems even more affectionate toward humans as a result of his ostracism from goat-kind. In fact, he's the most dog-like goat I've ever met. He definitely inherited all of Artemesia's sweetness and added a level of charm all his own.

In other words --- Punkin's new owner had better pick him up soon before Mark and I fall any further in love!

Joey chatter

I only clean my solar panels when they have a coating of ice or snow.

Anna (Anna and Mark: Waldeneffect)
The passage of time
Summer garden

Some days, the summer feels so fleeting.

Ripe sweet corn

It's time to harvest the first sweet corn...

Bare garden bed

...and plant the last sweet corn.

Five months from from frost to frost seems like an eternity in May and a heart beat in October. I'd better pause now and soak up the sun.

twenty years of free software -- part 13 past and future

This series has focused on new projects. I could have said more about significant work that didn't involve starting new projects. A big example was when I added dh to debhelper, which led to changes in a large percentage of debian/rules files. And I've contributed to many other free software projects.

I guess I've focused on new projects becuase it's easier to remember things I've started myself. And because a new project is more wide open, with more scope for interesting ideas, especially when it's free software being created just because I want to, with no expectations of success.

But starting lots of your own projects also makes you responsible for maintaining a lot of stuff. Ten years ago I had dozens of projects that I'd started and was still maintaining. Over time I started pulling away from Debian, with active projects increasingly not connected with it. By the end, I'd left and stopped working on the old projects. Nearly everything from my first decade in free software was passed on to new maintainers. It's good to get out from under old projects and concentrate on new things.

I saved propellor for last in this series, because I think it may point toward the future of my software. While git-annex was the project that taught me Haskell, propellor's design is much more deeply influenced by the Haskell viewpoint.

Absorbing that viewpoint has itself been a big undertaking for me this decade. It's like I was coasting along feeling at the top of my game and wham got hit by the type theory bus. And now I see that I was stuck in a rut before, and begin to get a feeling of many new possibilities.

That's a good feeling to have, twenty years in.

Joey git-annex devblog
day 405 more git development

Revisited my enhanced smudge/clean patch set for git, updating it for code review and to deal with changes in git since I've been away. This took several hours unfortunately.


List of feeds:

  • Anna: last checked (25 posts)
  • Anna and Mark: Waldeneffect: last checked (3341 posts)
  • Joey: last checked (124 posts)
  • Joey chatter: last checked (1602 posts; 1 new)
  • Joey git-annex devblog: last checked (373 posts)
  • Joey: blog: last checked (16 posts)
  • Jay: last checked (25 posts)
  • Dani: last checked (21 posts)
  • Errol: last checked (28 posts)
  • Maggie too: last checked (11 posts)a
  • Maggie also: last checked (42 posts)
  • Maggie: last checked (35 posts)
  • Tomoko: last checked (75 posts)
  • Jerry: last checked (28 posts)