Second-order complexity and agent programming
The thing I'm bumping up against a lot these days is second-order complexity. Not the complexity of a particular PR. That matters too, but there's a meta thing I worry about more: the complexity of operations, of the resulting engineering system. Yeah, we can write the code, no problem. But does this team have the capability to manage this additional knob, this additional mechanism?
One of the easiest escape routes is when we can completely automate the mechanism. But it has to be completely automated with respect to the system model of the whole company.
Suppose you want to add an automated performance regression test. The best version of this runs in CI and fails if it breaks. Because then you have a closed loop: the CI test fails, the developer who introduced the regression is forced to fix it, main stays green, the performance test stays green, and the desired function of the change is assured via mechanism design.
But what if the performance regression test is expensive? Okay, run it once an hour or once a day. But then who monitors that result? How can I gain confidence that in a month, six weeks, this thing isn't just failing in the background, getting ignored?
Sure, we can add alerts. But one-off alerts suffer from a similar problem. There has to be a system in place for ensuring the alerts feed back into a system for fixing issues. And building an alerting system like that is expensive, and it requires a certain capability level. We tend to do it for mission-critical stuff. For non-mission-critical stuff, where the alerts don't feed into an on-call rotation, it may not work. Which leads me to sometimes feeling like, well, maybe we just don't add the performance regression test for now, because I can't think of a way to completely automate it with respect to the system model of the company.
This is one place where AI doesn't help as much as we'd like. It helps enormously in general. But AI can't really help me that much with mechanism design at the system level of the company. I suspect it can help some. For one, it lets me search the solution space of closed-loop automations much more effectively, and that's something I'm going to keep plugging into. But even there, the right solution to this problem might be something like: don't rely on a slow performance regression test at all, build a collection of really good micro-benchmarks. AI can help me write those, but there's a lot of re-architecture and reorganization needed to get there. It's not something I can one-shot in Devin. I have to think about it and apply design judgment.
So it gets put in the backlog, even though AI dramatically reduces the cost of writing the code involved in solving the constituent problems.