git-annex devblog (Joey devblog)
day 584 matching S3 histories

I've been working on matching up the git history with the history from a versioned S3 export. Got sidetracked for quite a while building an efficient way to get the git history up to a certian depth (including all sides of merge commits) without reading the entire git log output.

The history matching is mostly working now, but there's a problem when a rename is exported to S3, because it's non-atomic on S3 and atomic in git, and so the histories stop matching up. This is not fatal, just results in an ugly git history with the right tree at the top of it. It's not entirely wrong; the git repo and the S3 bucket did legitimately diverge for a while, so shouldn't the merged history reflect that? The problem is just that the divergence is not represented in the opimal way.

I hate giving up at the final hurdle, but I feel I need to think about this some more, so merging import-from-s3 is postponed for another day, or likely until Monday.

Anna (Anna and Mark: Waldeneffect)
Gardening in young soil
Early lettuce

We've been in Ohio for a year and a half, but took us six months (most of that winter) to get the fence going so we could safely plant inside. Which means our vegetable garden is just about a year old.

Truck by garden

During that year, I've been learning local biomass sources. Most are higher-carbon and less-aged than I'd prefer, but they make up for what they lack in quality in their quantity and ease of hauling. When the road-clearing crew dumped a huge pile of wood chips right by my garden gate last year, I decided I love being close to a road!

Young garden soil

At the same time, I'm remembering what it was like to plant into young garden soil. Between cover crops and copious additions of organic matter, I'd gotten my Virginia soil primed so growing there was nearly like planting into big mounds of potting soil. Up here, the clay subsoil is close to the surface and the biomass I'm adding is only gradually working its way down through no-till layers of cardboard and newspaper.

Side-dressing with manure

So I garden a little differently. Applying half-composting horse manure/wood chips is acceptable in most garden areas (although the combo did kill a barely-survived-the-winter thyme plant). The trick is to pull the mixture back a bit from the plants (sidedressing instead of topdressing). If possible, I also apply manure at least a month before my planting date.

Vegetable seedlings

I've also been starting more seedlings inside, potting them up into small cups and letting them grow for a few weeks before setting them out into the garden. The reason for this is twofold.

Pea seedlings

First, it's tougher to get seeds to sprout happily in subpar soil (especially since I'm irrigating less now that we're on city water). Second, voles have moved in and are nibbling up young sprouts (especially my peas!). As you can see in the photo above, the ornery rodents ate all of my direct-seeded plants (on the other side of the trellis), while they left older transplants alone.

Strawberry flower

Starting spring crops inside is also handy because the soil takes longer to warm up in Ohio. Our strawberries are just now starting to bloom, two or three weeks later then they tended to in Virginia. That means annuals would also be two or three weeks behind...if I hadn't jumped the gun with early seeding indoors.

Free garden biomass

Of course, that's all short-term fixes that will be in the rear-view mirror in a few short years. To boost our soil's fertility fast, I'm layering huge quantities of organic matter in areas I won't be planting into for several months.

First came the spoiled hay (left) from a local farmer --- she had more of it than I could handle, but I piled up as much as I could to start building the soil. Aisles of wood chips (middle) will feed fertility more slowly, while new beds made of deeply mounded horse bedding/manure (right) will rot down within a few months.

Bathtub worm bin

As if that's not enough, Mark is building me worm bins out of old bathtubs. We'll fill these up with horse manure as well (so easy to accumulate now that it can be scooped into our truck then unloaded directly into our garden on the other end!). By this time next year, we should be overflowing in good, rich compost to feed our dirt.

git-annex devblog (Joey devblog)
day 583 S3 import and export fully working

Got S3 import and export working fully for both versioned and unversioned buckets. This included developing a patch to the aws library; only versioned buckets are fully supported until that gets merged.

I'm left with one blocking problem before merging import-from-s3: The commit history when importing from a versioned bucket is too long. It needs to find the point in the versioned import that has already been committed and avoid committing it again. Have started on that, but didn't get all the way today.

Also, this S3 import feature should be able to be used with anonymous S3 access to a bucket, and indeed that might be more common than wanting to import from a bucket you own or have credentials to allow access to. But the S3 remote does not currently try to use anonymous S3 access, so supporting that will need some more changes.

(Keyboard is fixed, yay!)

git-annex devblog (Joey devblog)
day 582 versioned S3 import working

Despite struggling with a keyboard controller that's increasingly prone to flaking out and not registering some key presses while doubling others, I managed to finis implementing import from versioned S3 buckets. It's quite nice to see it download past versions of files and construct a git history.

Still enough unimplemented stuff and bugs to need to work on this for probably one more day.

(Imagine here me stuggling for a full minute to :wq)

git-annex devblog (Joey devblog)
day 581 starting import from S3

Started today on git annex import from S3, in the "import-from-s3" branch.

It looks like I'm going to support both versioned and unversioned buckets; the latter will need --force to initialize since it can lose data.

One thought I had about that is: It's probably better for git-annex to be able to import data from an unversioned S3 bucket with caveats about avoiding unsafe operations (export) that could lose data, than it is for git-annex to not be able to import from the bucket at all, guaranteeing that past versions of modified files will be lost. (Rationalization is a powerful drug.)

To support unversioned buckets, some kind of stable content identifier is needed other than the S3 version id. Luckily, S3 has etags, which are md5sum of the content, so will work great. But, the aws haskell library needs one small change to return an etag, so this will be blocked on that change.

I've gotten listing importable contents from S3 working for unversioned buckets, including dealing with S3's 1000 item limit by paging. Listing importable contents from versioned buckets is harder, because it needs to synthesize a git version history from the information that S3 provides. I think I have a method for doing this that will generate the trees that users will expect to see, and also will generate the same past trees every time, avoiding a proliferation of git trees. Next step: Converting my prose description of how to do that into haskell.

Anna (Anna and Mark: Waldeneffect)
April mushroom foray
Windy fox

I've been on some pretty amazing adventures since I checked in last. The most inspiring was a five-day wolf-watching trip in Yellowstone National Park (which I blogged about on my werewolf site because, well, wolves).

Mushroom foray

More relevant to this blog is the mushroom foray I attended yesterday in Wildcat Hollow outside of Glouster. This was such a delightful adventure, like an Easter egg hunt in the woods searching for every kind of fungus we could find to be ID'ed by pros then uploaded to a website for inventory purposes.

Devil's urn mushroom

Most of the mushrooms we found were just pretty, like this Devil's Urn. Some weren't even pretty --- drab and dried fruiting bodies from last year. And then there were the medicinal/edibles, about which I took extensive notes.

Dryad's Saddle

I'll start with the Dryad's Saddle, which I'd seen many times before. But I'd never understood that the young ones are such choice edibles, identified both by their unique visuals and also by their even more unique cucumber scent. Chop up any that are tender into thin slices then saute for about ten minutes until well cooked,and I suspect you'll find them as complexly delicious as Mark and I did.

Wood ear mushroom

The Wood Ear is used widely in Asian cooking, so I might have actually eaten this one before without knowing it. It's a jelly-like fungus, growing on live wood. I haven't cooked my sample yet --- tomorrow's experiment!

Deadly Galerina

Do not, however, nibble on these Deadly Galerinas. Their orange-tan color is relatively distinctive and they grow in a similar habitat to the edible Honey Mushrooms. If in doubt, throw it out!

True and false turkeytails

Moving on to medicinals, I enjoyed a side-by-side comparison of False and True Turkeytails, finally wrapping my head around the differences. True Turkeytails are polypores, fuzzy on top and rough on the bottom. False Turkeytails are parchment mushrooms, smooth on both sides. These images are last year's fruits, so both species have faded a lot. But I now feel confident I could pick the True and not the False.

Mushroom identification

The image above is just a small sampling of our haul, shown in all its glory. Martha from the Ohio Mushroom Society rattled off scientific names so fast most went in one ear and out the other. But I joined up and will look forward to learning more during our next foray!

Slug and toadshade

I can't resist ending with at least one of the many stunning wildflowers that graced this trail. Go outside --- it's the most beautiful time of the year!

git-annex devblog (Joey devblog)
day 580 import from android

It was not very hard to get git annex import working with adb special remotes. This is a nice alternative to installing git-annex on an Android device for syncing with it. See android sync with adb.

I'm still thinking about supporting import from special remotes that can't avoid most race conditions. But for adb, the only race conditions that I couldn't avoid are reasonably narrow, nearly as narrow as git checkout's own race conditions, with only the added overhead of adb. So I let them slide.

Today's work was sponsored by Trenton Cronholm on Patreon.

Maggie too
Oregon ho!

A lot of people who are not super familiar with Oregon geography might be surprised when I announce my excitement that I am heading to a desert mountain-range in a county that's seat boasts a population of just 2,000.  I literally think I will be more likely to run into a rattlesnake or a fossil of some prehistoric dog-bear than another human being.  Fortunately, my best friend and long time pen pal and her boyfriend will also be present.  When I think of how remote her home must be, it starts to make more sense why my friend is so happy to have me in her life, when so many other close people estrange me due to "drama" as they say or "over-exuberance" as I recant.  The fact that I feel such kinship with my old friend from college is one of the main reasons I have decided to cross the country and then fjord half the Western state, rather than immersing in a wetter climate.  But I am starting to get excited about this new to me bioregion and the unique microclimate I am preparing to behold.  In my friend's letters she always paints such an amazing description of her farm landscape, including but reaching far beyond rattlesnake skins that seem to be everywhere.  It is these interesting plant names that I never have heard of "dry grasses," "various sticks," and "poms of dried yarrow calyxes" or rather I have heard of some of these but I have never set foot in a desert and this is a place so seldom visited it deserves a field guide and write up in some travel book.  But don't rush out all at once!  I am kind of thrilled to be breaking way on my Oregon trail.  So I'll keep you posted as things progress.  

Maggie too

From birth we were taught how to bite 

hard like a wolf.

You can't softly kiss an elk

and expect for it to roll over

offering itself for you to eat it.


Maggie too
wolf wolf wolf
rainy day
rainy day
arch of the feline
arch of the feline
Can you see a wolf in the stump?
Can you see a wolf in the stump?
can you see a wolf on the bench?
can you see a wolf on the bench?

Maggie too
Relearning How to Swim

Three days ago, I went to the pool to swim laps in the big pool, but something in the small pool tempted and drew me.  I lowered myself into the water, wondering why I was there.  

Down and up the lane until I noticed on the floor of the pool, an object ovular and green.  I jump dove into the deeper bottom waters discovering a diving ring.  Then, like a child, I began playing with it.  

There is something about lapping that is more mature, less playful.  For most of my life, I have loved swimming the pattern of laps, but lately I have grown supremely bored with it.  I just couldn't get my blood pumping fast enough.  Had nearly given up.

Playfulness adds something to my swim.  I threw the ring in front of me in the lane, and each lap I dove for it, then I'd throw it again.  My arms felt themselves doing something different, a variety important to my movement.  

My feet and legs bounced off the ground to gain momentum for diving deep.  Then I acquired two additional rings from my friendly lifeguard, and joyfully, I started to juggle.  I held one ring in one hand, two in the other, and swimming front crawl, I'd throw one ring each time my arm went forward.  

At first it was hard to figure out the pattern of swim juggling.  The whole first day, I tried to juggle, but ended up spending much of my time gathering the rings from the pool bottom.  I transfered to the deep pool with three lap lanes, and played and played.  But the second day, when I got three rings from where I had noticed them, juggling was much easier.

Overnight, my body had learned how to water juggle, or at least my attempt was drastically improved.  It also bares mentioning that I was having an insane amount of fun, that I no longer was bored with my swims, and also, my heart was pumping wonderfully.  I have played with rings three days now.

It feels like my child side is teaching me how to live again when I play with the rings in the water.  I move my body in the strangest positions doing what I do, and my heart is getting the work out it needs.

I'd recommend playfulness to anyone who's workout gets stuck in a rut.  Mine had been for a long time, and now I am eager to get to the pool again.


List of feeds:

  • Anna: last checked (50 posts)
  • Anna and Mark: Waldeneffect: last checked (4533 posts)
  • Joey: last checked (185 posts)
  • Joey devblog: last checked (208 posts)
  • Jay: last checked (50 posts)
  • Errol: last checked (53 posts)
  • Maggie: Cannot detect feed type (35 posts)
  • Maggie too: last checked (72 posts)
  • Maggie also: Can't connect to (No suitable addresses found) (276 posts)
  • Tomoko: last checked (77 posts)
  • Jerry: last checked (28 posts)
  • Dani: last checked (22 posts)
  • Richard: last checked (41 posts)