Great presentation and well articulated. It's nice to see Sourcegraph publish this type of content, acknowledging the major shortcomings of LLMs when placed within real-world development environments. My only critique is that the computer vision demo felt contradictory to the foundational premise of the deck (i.e., 'LLMs excel at greenfield dev and suck at real-world dev... now watch yet another demo of an LLM doing greenfield dev'). It would be more interesting to see Sourcegraph deploy Cody within a 1,000-file codebase comprised of a frontend, backend, and infrastructure-as-code repo, pinpoint needles in the haystack, and facilitate full-stack feature development without becoming overwhelmed or lost. 20 files worth of context would not suffice, and an additional layer of 'pre-training' on the codebase architecture would surely be needed... Perhaps this sort of performance is only available to enterprise companies with deep pockets?
coooooooL
Great presentation and well articulated. It's nice to see Sourcegraph publish this type of content, acknowledging the major shortcomings of LLMs when placed within real-world development environments. My only critique is that the computer vision demo felt contradictory to the foundational premise of the deck (i.e., 'LLMs excel at greenfield dev and suck at real-world dev... now watch yet another demo of an LLM doing greenfield dev').
It would be more interesting to see Sourcegraph deploy Cody within a 1,000-file codebase comprised of a frontend, backend, and infrastructure-as-code repo, pinpoint needles in the haystack, and facilitate full-stack feature development without becoming overwhelmed or lost. 20 files worth of context would not suffice, and an additional layer of 'pre-training' on the codebase architecture would surely be needed... Perhaps this sort of performance is only available to enterprise companies with deep pockets?