[!note] a multi-threading deadlock in a database vendor’s ODBC driver. She had to manually disassemble the vendor library and then generate a binary patch. By hand. (Location 421)
- was in awe
[!note] She led the effort to systematically isolate, diagnose, and restore those services based on decades of intuition and countless production battles. (Location 433)
- eschews:: Avoid
- despises:: 鄙视
[!note] applying the strangler pattern to dismantle decades-old code monoliths and replacing them safely, confidently, and brilliantly. (Location 441)
[!note] She doesn’t want to see brochures for the car they’ve promised to build—she wants to see the engineering plans and the actual parts that they’re going to assemble the car from. (Location 469)
[!note] Find someone who’s actually doing integration testing so I can shoulder-surf while they work. (Location 479)
- succinct:: 简洁
[!note] Millions of Word docs filled with manual test plans. (Location 513)
[!note] need a small army. (Location 515) We have a test army for SA, exactly
[!note] a bunch of contractor developers brought in two months ago. They can’t do builds, either. Shocking. I took them out to lunch. What a disappointment. They know even less than I do. (Location 525) 太上头了
- argle-bargle.:: 讨价还价
- “Yes, we’re definitely behind, but we have a couple of new developers starting this week to help, and they should be up to speed and productive in a week or two.”:: Have you ever read the mythical man month??
[!note] We need you focused on features.” (Location 731) WTF
New highlights added December 28, 2022 at 4:11 PM
[!note] Kurt Reznick, QA Manager (Location 49)
[!note] Dwayne Cox, Lead Infrastructure Engineer (Location 51)
[!note] Brent Geller, Ops Lead (Location 52)
[!note] William Mason, Director of QA (Location 55)
[!note] Wes Davis, Director of Distributed Technology Operations (Location 56)
- Kirsten Fingle, Director of Project Management
[!note] Bill Palmer, VP of IT Operations (Location 63)
[!note] Developers cannot be productive without a great build, integration, and test process. (Location 667)
[!note] Maybe it’s because when friends do favors for friends, we don’t require them to open a ticket first. (Location 928)
[!note] Functional programming is truly a better tool to think with. (Location 993)
[!note] So, when this started happening to her project, she spent every evening for weeks rearchitecting her system so that people could quickly, easily, and safely make the changes they wanted. (Location 1030) I like this little story about Maxine’s side project. As an architect, you should define the structure of your code, set the standard, and provide examples and documentations, what you should not do is control very piece of the code in detail. Contributions should be encouraged, but guided.
[!note] Maxine knows that agility is never free. Over time, without this type of investment, software often becomes more and more difficult to change. (Location 1033)
[!note] But in almost every other domain, especially when you have customers, change is a fact of life. (Location 1036)
[!note] able to build things with focus, flow, and joy. (Location 1052)
[!note] People were able to do what they wanted without being dependent on scores of other people. This is what great architecture enables. (Location 1053)
[!note] Maxine does not delight in the suffering of others, but watching the fireworks surrounding Phoenix is much more exciting than waiting for people to work on her tickets. (Location 1070) How to say 幸灾乐祸in English
[!note] She grins again, not caring. (Location 1083) 幸灾乐祸脸
[!note] William, (Location 1086)
[!note] Kurt, (Location 1087)
[!note] Bill (Location 1087)
[!note] thoughtful capacity-planning exercise. (Location 1110) We need to do capacity planning. SRE
[!note] “How many transactions per second are we expecting for product displays and orders? And how many transactions per second are the current builds capable of handling right now? That will tell us how many servers we need for the horizontally scalable portions, as well as how far we’re off for the vertically scaled components, like the database.” (Location 1113) This is something that are from the architect’s vocabulary
[!note] Since when have you ever cared what people in authority think about you? (Location 1146) This should be the attitude of an architect!!! Never care about the people in authority think about you!
- Lieutenant Saavik expression,
[!note] Maxine is very picky about who she associates with. (Location 1168)
[!note] She is disheartened by how Parts Unlimited executives, the bridge officers, are so disconnected from the daily work of the “redshirts” in the technology organization. (Location 1224)
- abdicated:: 退位
[!note] eighty-page document full of tabs. (Location 1237) How to build a complicated system… But sometimes this is important and useful. With extensive build support, engineer are spoiled sometimes and they do not really care about building infrastructure but only the features! They lose the ability to build something from scratch.
[!note] Just scanning the section headings makes her heart leap—they’re the painstakingly assembled Phoenix build instructions, complete with links to documents, license keys, step-by-step tutorials, and even links to a bunch of videos. One is titled “Getting your uberjar to run in our (very) crazy, screwed up production web cluster (8 min),” and another is “How to monitor your apps despite our Ops groups (12 min).” (Location 1237)
[!note] Best of all, there’s a link to a four-node virtual machine cluster with administrative access! That means Maxine will be able to do whatever she wants without having to fill out another service desk ticket! (Location 1241)
[!note] But she can definitely see the different personalities and tech stacks of the different teams working on Phoenix. She had no idea there were so many. (Location 1283)
[!note] Maxine isn’t usually a fan of rigid standardization, but she’s not a fan of everyone getting to choose whatever they fancy in the moment. (Location 1286)
[!note] Each decision is a commitment to support it for years or even decades—these are decisions that go far beyond just one team. (Location 1287)
[!note] Kurt? He’s not the sharpest knife in the drawer, which is why he’s stuck in QA. He’s also really nosy. Why do you ask? (Location 1305)
[!note] modernize our architecture, platforms, and practices.” (Location 1321)
- cul-de-sacs,:: 死胡同
[!note] “Oh, by the way, sorry about that episode with John. He’s such a tool. But he’s my boss.” (Location 1367)
[!note] Everything is by committee. No one can make decisions, and implementing even the smallest thing seems to require consensus from everyone. (Location 1421) The importance of bottom up!!!
[!note] “I want to build and maintain something for the long haul, instead of shipping the ‘feature of the day’ and dragging all this technical debt around.” (Location 1427)
[!note] You clearly don’t understand how this game works. We’re QA. We protect the organization from developers. (Location 1453) OMG, is this really a QA directors’ thought? I am scared of this out-dated engineering culture.
[!note] You give developers an inch, and they’ll take a mile.’” (Location 1455) QA, SRE and Devs need to have a common OKR - improve the effectiveness of the CI/CD pipeline
[!note] “He only cares about growing his union membership dues, not about what’s right for the business. (Location 1461) That is bad..
- Despite the crisis, bean counters are still bean counters,
[!note] with people who, I’m sorry to say, are the people who didn’t have enough experience to be application developers.” (Location 1505) What a pity but sometimes that is reality
[!note] into Excel (Location 1586)
[!note] it. A BOM is an invisible first character that some programs put in a CSV file to indicate whether it’s big-endian or little-endian. She’s been bitten by this before. (Location 1590) Me as well as someone worked with csv for a very long time!!
[!note] “You have no idea,” Brent says, rolling his eyes. “Different problem every time, depending on who generated the file. The most common problems lately are zero-length files, or files with no rows in them. And it’s not just the pricing team—we have data problems like this all over the place.” (Location 1596) We need a sophisticated csv parser program here!
[!note] The Phoenix rollout was more than just a spectacular outage … it was the most amazing example of production data loss Maxine has ever seen. (Location 1614)
[!note] Maxine stares at him, speechless. He’s still working on features for the next release. (Location 1653) Speechless, this is the type of people should be laid off…
[!note] Maybe we explore outsourcing IT? (Location 1664) Speechlesssssssss I hate contractors !!!!!!!
[!note] “I wasn’t a huge fan of William,” he says, “but to blame him for everything is wrong.” (Location 1757) William is the QA director that does not believe in test automation, because without automation, he can get big “budget”….
[!note] “You’d think we have thirty teams of ten people, with each team able to get things done independently. But at times, it’s like we have only one team of three hundred people … Or maybe three hundred teams of one. In either case, something is very wrong … (Location 1812)
[!note] “You should be able to create value by changing one file, one module, one service, one component, one API call, one container, one app, or whatever! (Location 1918) Create value by changing one thing!
[!note] “Build responsibility moved from Dev to QA to interns. Tech giants like Facebook, Amazon, Netflix, Google, and Microsoft give Dev productivity responsibilities to only the most senior and experienced engineers. But here at Parts Unlimited, it’s the exact opposite.” (Location 1928)
[!note] “Sometimes it’s when simple components become complected, and you can no longer reason about it or change it without immense effort or risk of catastrophe. Sometimes it’s when decision-making processes or the organizational structure loses locality, forcing even small decisions to be escalated—your infamous ‘Square.’ (Location 1937) I like the word - locality!
[!note] “I’ve started calling all of these things ‘complexity debt,’ because they’re not just technical issues—they’re business issues. And it’s always a choice,” (Location 1940) Tech debt and product debt, and biz debt!!! Are the team organized in an effective way? Are the processes optimized?
[!note] “You can choose to build new features or you can choose to pay down complexity debt. When a fool spends all their time on features, the inevitable outcome is that even easy tasks become difficult and take longer to execute. And no matter how hard you try or how many people you have, it eventually collapses under its own weight , forcing you to start over from scratch.” (Location 1941) Restart from scratch…
[!note] the First Ideal of Locality and Simplicity. (Location 1973)
[!note] “The Second Ideal is Focus, Flow, and Joy. (Location 1977)
[!note] The Third Ideal is Improvement of Daily Work. (Location 1986)
[!note] The Fourth Ideal is Psychological Safety, where we make it safe to talk about problems, because solving problems requires prevention, which requires honesty, and honesty requires the absence of fear. (Location 1987)
[!note] Fifth Ideal is Customer Focus, (Location 1989)
[!note] The First Ideal—Locality and Simplicity The Second Ideal—Focus, Flow, and Joy The Third Ideal—Improvement of Daily Work The Fourth Ideal—Psychological Safety The Fifth Ideal—Customer Focus (Location 1992)
[!note] QA is often viewed as an underclass, but at least they’re above Ops. All of which is crap (Location 2011) Change your perception of QA role
[!note] one-month feature freeze. (Location 2052) Here comes! The feature freeze…
[!note] “This is our moment to shine. Here’s our chance to show people what engineering greatness looks like,” (Location 2056)
[!note] byzantine processes (Location 2069)
[!note] It is almost impossible to predict how a program will behave if any other part of the program can change data that you’re depending on at any time, (Location 2139)
[!note] Another testament that functional programming principles are better tools to think with. (Location 2160) Maybe a good time to read Java 8 book again? Or study typescript to the best
[!note] “Nope, not any more. We used to be able to do that too. But now someone else deploys it for us. ‘Stay in your lane,’ they told us.” (Location 2174) Let Dev test, let Dev deploy!
[!note] That’s just not good enough for her. In her MRP team, any developer could test their own code and even push code into production themselves. (Location 2192)
[!note] In her MRP team, any developer could test their own code and even push code into production themselves. They didn’t have to wait weeks for other people to do that work for them. (Location 2192) We need to enable the Devs of SA to do that!
[!note] Amazon ideal of the “two-pizza team,” where features can be created by individual teams that can be fed with two pizzas. (Location 2213)
[!note] “So, someone has to respond to the ticket and copy the logs off the server for us?” (Location 2235)
[!note] Four hours later, after reviewing the production logs, they confirm that the problem isn’t Data Hub. Two hours after that, everyone finally agrees. As Tom had suspected, it was an internal networking (Location 2246) Remind me that i spent an hour to dig into various applications metrics and logs to indentify a low level service mesh infra issue(networking connection issue), in order to conduct a root cause analysis of a production issue. I am totally fine with my current team and company!
[!note] Chad Stone in network engineering is no longer with the company. (Location 2255) Holy shit…fire anyone who made mistakes…
[!note] But over the years, we’ve hard-coded some things that we shouldn’t have, updated some things here, updated other things there, never quite documenting all of it … and now? It’s a mess.” (Location 2274) Hardcoding No documentation ….
[!note] “We need developers to be able to focus their best energies on building features, not trying to get builds to work. I have a ton of passion for this too and would love your help. (Location 2292)
[!note] ‘all the idle developers’ (Location 2300)