181
submitted 9 months ago* (last edited 9 months ago) by lemmyreader@lemmy.ml to c/asklemmy@lemmy.ml
you are viewing a single comment's thread
view the rest of the comments
[-] neidu2@feddit.nl 13 points 9 months ago* (last edited 9 months ago)

No. But I am owed a beer for every project I show up to. Happens around twice per year.

Before a project that involved outfitting a survey ship headed for the Mediterranean, this conversation took place: (paraphrasing)

Me: This storage cluster is based on 10gig/s network. I don't think the data processors will get the throughput they require.

Chief tech: I tallied up the data transfers, and the time available should suffice with 10g/s.

Me: Did you account for the fact that they duplicate the data and transfer a lot of it twice?

Chief tech: No. But I'll tell them not to do that.

Some time goes by. Project starts, 1PB of data is collected, and processing is starting up, scheduled to complete everything shortly before they arrive back in the home port. Then my phone rings.

Project manager: Throughput speeds are too slow. Is there anything that can be done, or do we have to eat the daily fine for being too late?

Me: How much is the daily fine?

PM: (number) USD

Me: I'll make the purchase req now for one tenth of the daily sum. That should increase the throuput.

PM: Hardware install involved?

Me: Yes. I'll order, configure, and document everything for the chief tech to plug&play during their refuel stop.

PM: I owe you a beer for the next projects.

The extra cost wasn't that much, considering it was the same upgrade I recommended before project startup anyway. We just had to pay for express freight from the vendor to my home, and from my home down to the ship. Long story short, it worked out. A Mellanox 2100 was shipped down to Malta, and installed by the crew, and it worked out of the box. I was worried about the extensive VLAN and LACP setup in play. But as that setup was my design, I managed to wing it when preparing this switch. I had been using the production cluster for some testing earlier, so it already had the necessary kernel modules for the 100gig cards. The processors had everything wrapped up and written to tape the day before arriving back home. The night of arrival there was a project completion dinner for the crew and us field service personnel. I drank for free that night. I don't remember it all, except that it was fun, and that the only reason why I called it a night at 0300 was because the hotel bar was closed.

To the chief techs defense, he wasn't that wrong. He'd made a reasonable estimate based on the numbers he had available. Unfortunately he didn't have all the numbers, as well as give the processors some head room for fudge factor. This chief tech was mostly experienced in only providing raw dataand thus working without the processing aspect.

[-] lemmyreader@lemmy.ml 2 points 9 months ago

Thanks for this story, I enjoyed reading it.

[-] Kaliax@lemmy.sdf.org 2 points 9 months ago

I enjoyed reading this, hopefully you share more stories on Lemmy in the future.

[-] neidu2@feddit.nl 2 points 9 months ago

I probably will when relevant

this post was submitted on 22 Jan 2024
181 points (96.4% liked)

Asklemmy

43880 readers
1560 users here now

A loosely moderated place to ask open-ended questions

Search asklemmy ๐Ÿ”

If your post meets the following criteria, it's welcome here!

  1. Open-ended question
  2. Not offensive: at this point, we do not have the bandwidth to moderate overtly political discussions. Assume best intent and be excellent to each other.
  3. Not regarding using or support for Lemmy: context, see the list of support communities and tools for finding communities below
  4. Not ad nauseam inducing: please make sure it is a question that would be new to most members
  5. An actual topic of discussion

Looking for support?

Looking for a community?

~Icon~ ~by~ ~@Double_A@discuss.tchncs.de~

founded 5 years ago
MODERATORS