A coworker is trying to get everyone at work to just vibe code everything and today in a meeting he said something along the lines of "I know today that we rely on internal expertise, but I don't think we should do that anymore". Buddy if we can't rely on internal expertise how the fuck are we supposed to validate the output of these LLMs? We can't trust the LLM to validate itself. It was faking the tests in his PR. It wrote dozens of tests that asserted nothing, but he didn't see a problem with that because the test coverage was higher than average.
Timeline
Post
Remote status
Context
1For those unaware: If you write a series of tests that assert nothing, then the coverage numbers don't fucking matter. You could write tests that get you 100% test coverage, but don't fail when you break the code. That coverage number is only as good as your tests.
Replies
0
No replies yet.