c0t0d0s0.org

If it weren't about the AF447 disaster, some comments of readers in some of the forums were actually funny. Especially those ones writing that Air France just had to install GPS in addition to pitots to indicate the speed of the aircraft when the pitot tubes got unreliable due to icing. After all the el-cheapo navigation system for the car shows the speed quite well. Well, while the speed on the GPS display is pretty precise, it has almost no value for keeping the aircraft in the air.

As you you may from school aircrafts are keep in the air by a force called lift. And aircraft has to create at least enough lift force to neutralize the force of gravity. The lift is dependent on the speed you move the airfoil through the air. Okay, that's vastly simplified but sufficient for my explanation. Dependent on the aircraft design the needed speed to create such an amount of lift is different: For example the Fieseler Fi156 has an incredible low minimum speed: 27 knots. Out of this reason this aircraft is capable to fly backwards in really strong winds. But this backwards flying (negative speed if you want to say so) is just in relation to ground … the airspeed is still positive. The difference between the speed of the wind and the speed of the aircraft have just to be larger than the minimal speed of the aircraft and the wind speed must be larger than the aircraft speed. Don't believe me … just go into the next harbor and look at the gulls. Of course you won't see a Boeing or an Airbus backwards … at least not with usual winds.

Another nice example are aircraft carriers: Those have really large engines to accelerate to and to keep top speed. Out of a reason: When an aircraft carrier moves with 30 knots, the air on the flight deck on the aircraft carrier already moves along the airfoil with 30 knots even on calm day. Let's assume a fresh breeze (6 beaufort) or 30kn. When the carrier steers in the direction with the wind, the air would move along the airfoil … well … not at all. When the carrier goes in the direction against the wind, the air would flow along the airfoil with 60 kn.

That's the reason why an aircraft carrier turns into the direction against the wind to start aircrafts. It gives the aircraft a headstart in order to generate lift. And to get an aircraft to 30kn you need really big engines.

To get back to the difference between airspeed and speed over ground? What's the speed over ground? 30 knots. In all cases. But you have three different air speeds. The GPS only measures the speed over ground. However as mentioned above the lift - the force keeping the aircraft in the air - is dependent of the air speed. At 0 airspeed no lift. At 60 kn a lot of lift. At 30 kn airspeed, something in between. And then you have the jet engines and the catapult.

Speed over ground is something different than air speed. And a GPS can't measure air speed. The air speed can only be measured by measuring the speed of the air flowing along the aircraft. To do so, modern aircrafts use so called pitot tubes in conjunction with measuring equipment to measure the static pressure (the ambient air pressure). The pitot tube measures the ram air pressure into the pitot tube. If there is something blocking the air from pushing into the probe, it can't measure the speed.

So … a GPS receiver would have helped the pilots of AF447 not a bit. There are other means like the Backup Speed Scale but that is working completely different.

In customer projects, i'm talking more and more frequently about solid state disks and their performance. From this point the release of a testing specification for SSD performance is really welcome to me, as you often compare apples with pears and pears with potatoes. Looks like the SSD market gets more mature.

Darren Moffat explained in his blog how to encrypt swap and /var/tmp in Solaris 11 Express.

Ryanair? I will never ever put a foot in one of their aircrafts. Even when I have to book an private jet instead … I will never ever put a feet into one of their aircrafts. Period.
The flight back to Bremen was my 250th flight since loosing my fear of flying five - maybe six - years ago. I experienced a lot of things on my journeys … an aircraft from FRA to TXL that turned back to FRA just short of TXL because of inoperational slats. Unfriendly stewardesses. A go-around once. Approaches giving a roller coaster a run for their money. But yesterday was something completely new to me.

I have to explain something different first: For all the work and the hassles with helping me with the construction of my new home, I invited my mother, two of my sisters and my brother and his girlfriend to London for a one-day trip (my father can't use an aircraft out of health reasons). I found a reasonable cheap ticket at Ryanair and with the flight pricing at other carriers such day would much more expensive.

There is something important to know to undestand the whole situation: My brothers girlfriend is from Chile, she is living in Hamburg for a long time now. So she is not an EU citizen We knew that you have to go to the check-in to get a stamp on the self-printed ticket when you are not a resident of the European Union. However when going through the boarding in Bremen this was an integral process of boarding along with the weighing of the hand luggage. They checked the documents, put the stamp on it and we were able to enter the plane without problems.

We had a nice day in London and at the end we went back to Stansted. It was a really great day. We were a little bit low on time however more than sufficient to do some last minute souvenir shopping, going through security, getting to the gate, albeit we were at the rear end of the row, boarding hasn't started at that time.

So my sisters and my mother went through the gate no much later, I went to through the gate and walked down in order to help my family (they don't fly that often), however when walking on apron I realized that my brother and his girlfriend didn't went with us. However I entered the aircraft to look after the rest of my family. But after an additional minute my brother and his girlfriend were still not on board. So what had happened? I had already an idea because as a experience troubleshooter, you immediately go through the possible failure modes of an situation. The stamp. Dang. But at that moment I thought "Well, they will check that and put the stamp on it when all people have boarded in order to speed up boarding". However when they closed the doors, I quickly called my brother (call ended before push-back) and I was right with my foreboding.

The girlfriend of my brother wasn't allowed to enter the aircraft because this stamp was missing. Just to explain it again: It's not an official stamp, it's a stamp the Ryanair visa counter put onto the ticket just to show, that you have showed them your passport that you have the valid visa. She had all the needed documents. Nothing was missing, nothing was unclear. (The proof : She flew with Easyjet to Hamburg today - with no hassles .. with no problems).The stamp was missing, they denied boarding, they closed the gate, and I flew with my mother and my sisters to Bremen. Of course my brother stayed in London as he couldn't leave his girlfriend behind.

Yes, we made the error not to go to the Visa counter as we thought in stress that the persons in front of the security have the same role as in Bremen. And so I have no hope to write a complaint that will yield more than a generic mail.

However the behaviour of "Don't give a shit, if you have to enter the flight as you need to get home and your group is already on the plane" attitude, the denied boarding because of a missing stamp Ryanair put themself on the printed ticked as part of their processes, the merciless closing of the gate, the inability of checking the visa at the gate (in my experience the people at the check-in and at the gate are often recruited from the same group of people, so it's not uncommon to see the person from the check-in at the gate again). This is an unacceptable behaviour to me. It's just unfriendly and it just show that passengers at Ryanair are just sentient lifestock that they have to fly from A to B.

I think at any other airline we could have used, they would have checked it at the gate, perhaps after calling the supervisor, and allowed her to board. But not at Ryanair.
Yes I know it's my fault. I booked at Ryanair. I would never do that for when just flying alone: I'm prefering Lufthansa and Germanwings, but Easyjet or Air Berlin are good as well e.g. . However I was able to purchase six return tickets at 144 Euro and that looked as a good deal then (half of it was credit card costs as I don't own the only obscure card allowing you to pay without extra costs). I knew that I have to expect nothing that just gives you hint that you are considered as valued customer. You know that when you book Ryanair.

How did this story found it's happy end: We were limited to help from remote: My father and I were able to help. They drove at night by bus to London Victoria and at the end they were able to sleep in a bed instead of a airport chair (70 GBP), I was able to book an Easyjet flight at 1:30 AM for (160 Euros for 2 tickets). 1:30 because I drove my sis from Oldenburg home and then back to Lueneburg. To add insult to injury: As my brother had my car keys, I had to rent a car (61 Euros) in Bremen … however I was glad that there was still a rental car at 21:45 at this rather small airport.

This morning my brother and his girl friend used the train from St. Pancras to go Luton (don't know what this will add to the bill). Had to find out a lot of things this morning how to get them to the airport in Luton without the need to print something out. E.g. the EasyJet bus service to Luton don't have a "buy ticket at curbside" option according to the website) … dang. My brother and his girlfriend were rather jazzed by situation and fscking tired, so I helped them by phone to get though London.

BTW: A big thankyou to my manager Kristan that he gave me an additional day of vacation without asking questions … so I had time to work things out

Just rebooking the flight wasn't an option. Rebooking 100 Euros … per person. As they would have to go to the checkin to get the boarding passes (as they hadn't an portable printer): 40 Euros … per person. Nice extra 280 Euros for Ryanair due to denied boarding. You could even speculate, if this is the reason for such a draconian enforcement of rules. Don't know what they want to achieve with such an behaviour. However I can tell you, what they achieved: Ryanair will not get a single Euro from me …

My brother arrived in Hamburg he is driving to Bremen in order to fetch my car from the car parking. Hell knows what this will add to the bill as you don't know if you just rented the parking lot for 24 hours or if you actually purchased it for life …

All this because of a missing stamp. And perhaps you know understand why I would rather use a bathtub and a paddle to get from UK to Europe than to board a Ryanair aircraft. Ryanair lost 9 customers yesterday and I suspect some additional people in the family of my brothers girlfriend. And when just one reader is as disgusted from this behavior like me the time invested in this blog entry is a better investment than the time I would need to write complaint that would be finally futile at the end.

PS: As fate isn't without a sense of irony, the aircraft hat to go around at Hamburg and they flew circles for an additional 15 minutes because of problems with the runway in HAM. When it rains it pours.

Questions about deduplication are quite frequent. Not about the basic function, but about some implications of the implementation. Over the time i wrote a cheat sheet for myself to answer such questions.

• Deduplication in ZFS is in-band. The deduplication occurs when you write to the disk.
• The instrument to do deduplication is to use checksums. If a block has the same checksum as a block already written to the pool, it's considered as a duplicate and thus just a pointer to the already stored block is written to disk. To say it simplified)
• As it's not reasonable to scan through all blocks on disk to read the checksums in order find a duplicate, the checksums are stored in a structure called deduplication table or DDT.
• The DDTs are a so called AVL tree (that's obvious as there are lot of calls to the system wide avl tree code). This helps searching, so it doesn't just grep through the DDT. It's more intelligent.
• The DDTs are managed as a ZAP object on-disk
• ZAP is short for the ZFS Attribute processor
• A ZAP object stores key value pairs used in many tasks organizing data stored in ZFS
• ZAP data is metadata
• Metadata is cached by the ARC
• As it's cached in ARC, the ZAP blocks and the the data inside those blocks is object to the cache eviction policies of ARC
• Just increasing memory size in order to get more room for the DDT may be insufficient, as the space metadata can occupy in ARC is limited.
• Per default it's limited to 25% of the maximum ARC size (see line 3478 of arc.c)
• The maximum ARC size is 3/4 of the systems memory or all-1GB, whatever is higher.
• Both values are user-tunable.
• You can set the amount of memory for the metadata to a user-chosen value with zfs:zfs_arc_meta_limit= in /etc/system.
• That may be necessary when you know that you do a lot of deduplication tasks and the performance increase by keeping all or most of the DDT in memory outweights the loss of memory for caching data. For example mass importing data from a non-deduplicated storage may such a task.
• The size of the ddt may vary over versions, at snv133 the size of object is

  jmoekamp@hivemind:~# echo ::sizeof ddt_entry_t | mdb -k 
   sizeof (ddt_entry_t) = 0x178

  Or decimal - 376 Bytes.
• Such an entry exists for each unique block written to disk while deduplication is activated
• Thus blocks written while deduplication is deactivated are not getting deduplicates (obviously), however you can't deduplicate other locks with them as well
• So obviously a pool just using 128k blocks may need less memory to store the ddt than a pool just using 512 byte blocks
• Everything stored in ARC may be moved to the L2ARC. So an SSD may be used to increase the space to cache the DDT. Due to the nature of SSD (no head positioning latency) is a much better place to search for the nescessary DDT parts than a rotating head.
• However keep in mind that L2ARC doesn't come for free memorywise. Everything stored in L2ARC still leaves some information in the ARC. The relevant metadata structure is arc_buf_hdr_t (see line 431 of arc.c:

  jmoekamp@hivemind:~# echo ::sizeof arc_buf_hdr_t | mdb -k
  sizeof (arc_buf_hdr_t) = 0xa8

  Or decimal - 178 bytes.
• Just before you are asking about substituting a 376 Byte object by an 178 byte object - the ARC doesn't store a single dde entry in a ZAP block, they are storing multiple ones in it.
• The deduplication is not done when writing to the ZIL, it's done when writing it finally into the pool. Thus the deduplication is not done when the OS blocks for the write when doing a sync write, instead it's part of an async process afterwards. However it's possible that you will see write throttling kicking in, when the system isn't capable to to write all the dirty pages away timey.
• Access pattern for the blocks containing DDT data not in memory is very random.
• System with large pools with small memory areas will not work well, as each write will result in a larger number of reads in order to get the data from the DDT.
• The more unique blocks you have in the DDT, the less fast dedup will work, given a small amount of memory. A 1 gig memory in a the box and several TB of storage with very small recordsizes will be a sure way to yield horrible performance.
• Performance will be best, when the complete DTT fits in memory.
• Systems with medium memory size and SSD will yield good results
• However everything depends on the number of blocks writen after activation of the deduplication, as the size of the DDT.
• In short: Small pool, lot of memory - yeah, really big pool, minimal memory - nah.Truth is somewhat between that...