Monday, January 15, 2007

Miksi aina juuri minä?

By now almost everyone's seen the Helsinki Complaints Choir video. I'm pretty sure the popularity started with a mention in Neil Gaiman's blog a couple months ago. The Birmingham and Hamburg copycats have replied with poor responses so save yourself the trouble.

I speak quite a few languages including Nordic ones, but Finnish is a totally different animal. It's not even Indo-European. That video gave me more than a laugh; it's given me a chance at better communication with customers which I've been able to employ recently. Someone at $FinnCo is doing a Berti, ignoring the answers laid out in front of him and explained in painful detail. FinnGuy keeps coming back asking more pointless questions for no reason I can ascertain.

Miksi toiset eivät ole1?

Dear REC,

All new files are getting corrupted. We have only 1,3 million files and 150/450GB free on the RAID. $Well-KnownSymptom is not occurring.

This corruption appeared to start on early Monday morning, Nov 27. No changes made at that time.
I see it already. If you've been playing along this past year you probably see it, too. "No changes made at that time." Which means they, in fact, did make a change and decided not to write the "except for" phrase.

Dear FinnGuy,
Which nothing was it that you changed?


Came the answer: "Only the load balancing." He didn't think to send the logs so I had to ask for them as well.

Servers which are load-balanced all need to be set up in the same way The only non-hardware differences between the servers should be their names and IP addresses. Software configurations need to be the same. Otherwise it doesn't work. that's kind of the point of load-balancing.

Once that was sorted we came across a different problem. It seems by enabling load-balancing, a long-running headache of a file system problem appeared. There are ways around it but no fixes yet. I added this new finding about reproducibility to our Defect Ticket and passed on the way to make things work.

Berti$FinnGuy wrote back and sent in more FileMon logs. And server logs. I explained again that yes, those weird errors we're seeing there are indeed indicative of the known problem, and enumerated the litany of errors that FileMon will show. Most FASTIO_WRITEs will fail. The system will then drop down to the IRP_MJ_WRITE method and succeed. The solution is to store files in this manner, not that one, otherwise $SpecialDirectory will be b0rked and you won't be able to write anything.

After the weekend $FinnGuy was back. He'd run even more tests and amazingly he was seeing loads of FASTIO_WRITE failures! Even more incredibly, $SpecialDirectory was b0rked and now no files could be saved!

I gave him directions on how to fix $SpecialDirectory, gave him explicit instructions again on how to save (use the ATTACH_FILE button, not the NEW_FILE button which should be disabled in the UI), and that should've been it.

i have added a new FileMon log file from our File system drive. I made two tests while running FileMon. A .zip file which was succesfully opened, and .log file which was opened as HTML (error) message. Both seems to have FAILURE and BUFFER OVERLOAD. Can you see anything else, which is different?

Nope. I can't see a goddamned thing that's different here. You keep using the NEW_FILE button and it will keep causing the same fucking failure Every. Fucking. Time.

It's not just insanity; fuckwittedness is also doing the same thing repeatedly and expecting different results (quote attributed to, among others, Einstein, Kipling and Brown).

Today $FinnGuy wants a phone conference. He wants to show me this in action. Never mind the fact that I've not only seen it, I was the one who tracked it down for hours with a different customer so we could finally make it occur at will and show the programmers. Even the most angelic and patient of toilet-training parents get tired of seeing the new poops junior made.

He deserves a Root Cause: 17-Fuckwit.

Instead the ticket's being closed with a Root Cause: 01-Defect for the Defect because I need to up my RC-01 numbers. Cops go through this, having to list one crime as another completely different crime so that the LoweringCrimePromiseOfTheWeek can be satisfied.

For $MegaCorp, it's defects again. Management sent a memo around saying that there's not enough defect reporting, not enough tickets are being closed due to defects, and that we need to get those reported defect numbers up. Never mind that doing so completely distorts any picture we have about why the hell we're getting all these tickets to begin with, the reason for bothering to write a Root Cause to begin with. And too, never mind that a secondary Root Cause explanation code was added to our system to further enlighten us. Now we're pissing in the pool. Fuckwits.

1 "Why doesn't anyone agree with me?!" The title of the post is "Why always me?"

x-posted from HuSi, where there's a poll.


Post a Comment

<< Home

In compliance with $MegaCorp's general policies as well as my desire to
continue living under a roof and not the sky or a bus shelter, I add this:

The views expressed on this blog are my own and
do not necessarily reflect the views of $MegaCorp, even if every
single one of my cow-orkers who has discovered this blog agrees with me
and would also like to see the implementation of Root Cause: 17-Fuckwit.