I was mainly doing python with gpt4, but now im working on an android project, so kotlin. Gpt4 wasn’t much use for kotlin, especially for questions involving more than a couple files. Sonnet is crushing it though, even when I give it 2k+ LoC. I’d say I’ve done about 2 months of pre-llm work in the last week, granted I am no professional, just a hobbyist.
For programming it is Sonnet 3.5, there is no remotely close 2nd place that I have tried or heard of, and I am always looking. I personally don’t really have any interest in measuring them in other ways. But for coding, Sonnet 3.5 is in a distant lead. Abacus.ai is a nice way to try various models for cheap. Really, some sort of agent setup like mixture of agents that uses Claude and got and maybe some others may do better than Claude alone. Matthew Berman shows Mixture of Agents with local models beating gpt4o, so doing it with sonnet3.5 and others of the best closed models would probably be pretty great.
I don’t know about your specific issue, but I have found that it helps quite a bit to often start new conversations. Also, I have a couple of paragraphs explaining the whole idea of my project that I always paste in at the beginning of each conversation. I’ve not been doing anything terribly complicated or cutting-edge, but I haven’t come across anything yet that Sonnet hasn’t been able to figure out, although sometimes it does take me being very clear and wordy about what I’m doing and starting from a fresh slate. I’ve also found it helps a lot if I specifically tell it to debug with lots of logs. Then I just go back and forth, giving it the outputs and changing code for it.