GenAI in Production

GenAI as a field is evolving super fast and along with it the supporting infrastructure as well. In the past 2 years this has generally led to the following problems -

1. A hurdle to deploy applications into production due to gaps in critical infrastructure
2. ex : Observability is not yet a fully solved problem - there are multiple solutions with mixed adoption across the field.
3. Purchasable infra solutions are in super early stages which makes for a buy vs build dilemma to developers
4. Should you buy something and commit to it early knowing that at this stage the feature set is not completely consolidated and this framework might be replaced in a span of 6 months?
5. Or should you wait out for the field to consolidate and commit later knowing that this will add significantly to efforts, costs and delays in the short term to our developers?

I have developed GenAI apps for the last 3 years (and LLMs for last 6 years). I have spent much of time building, testing and imagining frameworks from first principles. I am opinionated on how to build strong foundations and ship reliable GenAI software.