LLM Financial Applications

I started by asking AI to do everything. Six months later, 65% of my agent’s workflow nodes run as non-AI code. The first version was fully agentic : every task went to an LLM. LLMs would confidently progress through tasks, though not always accurately. So I added tools to constrain what the LLM could call. Limited its ability to deviate. I added a Discovery tool to help the AI find those tools. Better, but not enough. Then I found Stripe’s minion architecture. Their insight : deterministic code handles the predictable ; LLMs tackle the ambiguous. I implemented blueprints, workflow charts written in code. Each blueprint specifies nodes, transitions between them, trigger conditions for matching tasks, & explicit error handling. This differs from skills or prompts. A skill tells the LLM what to do. A blueprint tells the system when to involve the LLM at all. Each blueprint is a directed graph of nodes. Nodes come in two types : deterministic (code) & agentic (LLM). Transitions between nodes can branch based on conditions. Deal pipeline updates, chat messages, & email routing account for 29% of workflows, all without a single LLM call. Company research, newsletter processing, & person research need the LLM for extraction & synthesis only. Another 36%. The workflow runs 67-91% as code. The LLM sees only what it needs : a chunk of text to summarize, a list to categorize, processed in one to three turns with constrained tools. Blog posts, document analysis, bug fixes are genuinely hybrid. 21% of workflows. Multiple LLM calls iterate toward quality. Only 14% remain fully agentic. Data transforms & error investigations. These tend to be coding tasks rather than evaluating a decision point in a workflow. The LLM needs freedom to explore. AI started doing everything. Now it handles routing, exceptions, research, planning, & coding. The rest runs without it. Is AI doing less? Yes. Is the system doing more? Also yes. The blueprints, the tools, the skills might be temporary scaffolding. With each new model release, capabilities expand. Tasks that required deterministic code six months ago might not tomorrow.

Large Language Models (LLMs) possess vast capabilities that extend far beyond conversational AI, and companies are actively exploring their potential. In a recent tech blog, engineers at Faire share how they’re leveraging LLMs to automate key aspects of code reviews, unlocking new ways to enhance developer productivity. At Faire, code reviews are an essential part of the development process. While some aspects require deep project context, many follow standard best practices that do not. These include enforcing clear titles and descriptions, ensuring sufficient test coverage, adhering to style guides, and detecting backward-incompatible changes. LLMs are particularly well-suited for handling these routine review tasks. With access to relevant pull request data—such as metadata, diffs, build logs, and test coverage reports—LLMs can efficiently flag potential issues, suggest improvements, and even automate fixes for simple problems. To facilitate this, the team leveraged an internally developed LLM orchestrator service called Fairey to streamline AI-powered code reviews. Fairey processes chat-based requests by breaking them down into structured steps, such as calling an LLM model, retrieving necessary context, and executing functions. It integrates seamlessly with OpenAI’s Assistants API, allowing engineers to fine-tune assistant behavior and incorporate capabilities like Retrieval-Augmented Generation (RAG). This approach enhances accuracy, ensures context awareness, and makes AI-driven reviews genuinely useful to developers. By applying LLMs in code reviews, Faire demonstrates how AI can enhance developer workflows, boosting efficiency while maintaining high code quality. As companies continue exploring AI applications beyond chat, tools like Fairey provide a glimpse into the future of intelligent software development. #Machinelearning #Artificialintelligence #AI #LLM #codereview #Productivity #SnacksWeeklyonDataScience – – –  Check out the "Snacks Weekly on Data Science" podcast and subscribe, where I explain in more detail the concepts discussed in this and future posts:    -- Spotify: https://lnkd.in/gKgaMvbh   -- Apple Podcast: https://lnkd.in/gj6aPBBY    -- Youtube: https://lnkd.in/gcwPeBmR https://lnkd.in/deaMsxZy 

Ask your LLM the following question: "How many zeros are in 0101010101010101101?". A typical LLM might hallucinate the answer because it’s just predicting tokens. Now let’s raise the stakes: "What’s the current stock price of Google, and what was its 5-day average at market close?" To answer this, most LLMs must: 1. Pause to call a financial data API 2. Pause again to calculate the average 3. Possibly pause once more to format the result That’s multiple tool calls, each interrupting the thought process, adding latency, re-sending the entire conversation history and increasing cost. Enter CodeAgents. Instead of hallucinating an answer or pausing after every step, CodeAgents allow the LLM to translate its entire plan into executable code. It reasons through the problem, writes the script, and only then executes. Clean, efficient, and accurate. This results in: 1. Fewer hallucinations 2. Smarter, end-to-end planning 3. Lower latency 4. More reliable answers If you're exploring how to make LLMs think in code and solve multi-step tasks efficiently, check out the following: Libraries: - https://lnkd.in/g6wa_Wm4 - https://lnkd.in/gcuf2u5Q Course: - https://lnkd.in/gTse8tTw #AI #LLM #CodeAgents

We've spent the last two years building retrieval infrastructure. Here's a pattern I've been noticing: Recursive Language Models (RLMs) are starting to make parts of the retrieval infrastructure optional. Why? Because they treat the LLM more like a project manager. Instead of reading everything at once, the model: • Breaks down the problem • Delegates sub-tasks • Collects findings • Synthesizes an answer This makes them surprisingly effective in 4 scenarios: 𝟭/ 𝗟𝗮𝗿𝗴𝗲 𝗳𝗶𝗹𝗲 𝗽𝗮𝗿𝘀𝗶𝗻𝗴 Traditionally we'd build: • Chunking • Embeddings • Vector search • Graph retrieval An RLM can often eliminate the entire stack. Simply give the agent access to a large file or directory and let it recursively extract the information it needs via code. This is very similar to how I currently do research. I dump large amounts of information into a single file and let the agent query it on demand. 𝟮/ 𝗖𝗼𝗺𝗽𝗹𝗲𝘅 𝗰𝗼𝗱𝗲𝗯𝗮𝘀𝗲 𝗰𝗼𝗺𝗽𝗿𝗲𝗵𝗲𝗻𝘀𝗶𝗼𝗻 Understanding a large codebase is a decomposition problem. RLMs can: • Map dependencies • Analyze architecture • Launch sub-investigations • Aggregate findings The model behaves more like a staff engineer coordinating reviews than a chatbot reading code. 𝟯/ 𝗟𝗲𝗴𝗮𝗹 𝗮𝗻𝗱 𝗳𝗶𝗻𝗮𝗻𝗰𝗶𝗮𝗹 𝗮𝗻𝗮𝗹𝘆𝘀𝗶𝘀 Think: • Contracts • Policies • Due diligence • Audit trails These workloads require consistent reasoning across thousands of documents. RLMs recursively verify, compare, and reason through large collections of files without forcing everything into a single context window. 𝟰/ 𝗗𝗲𝗲𝗽 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵 / 𝗟𝗟𝗠 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗕𝗮𝘀𝗲𝘀 This is the use case I'm most excited about. RLMs can: • Explore thousands of files • Build intermediate summaries • Follow multi-hop connections • Synthesize findings The result feels more like managing a team of research assistants. The key takeaway: RLMs won't replace RAG. They sit on the same spectrum. For simple problems, RLMs can replace retrieval entirely. For complex problems, RAG narrows the haystack. The RLM deeply reasons over what's left. This is where the combination becomes incredibly powerful. P.S. I break down Recursive Language Models, how they work, and why they often become cheaper than brute-force long-context approaches in Decoding AI Magazine. Check it out here: https://lnkd.in/dfj6U3a9

I’ve been building and managing data systems at Amazon for the last 8 years. Now that AI is everywhere, the way we work as data engineers is changing fast. Here are 5 real ways I (and many in the industry) use LLMs to work smarter every day as a Senior Data Engineer: 1. Code Review and Refactoring LLMs help break down complex pull requests into simple summaries, making it easier to review changes across big codebases. They can also identify anti-patterns in PySpark, SQL, and Airflow code, helping you catch bugs or risky logic before it lands in prod. If you’re refactoring old code, LLMs can point out where your abstractions are weak or naming is inconsistent, so your codebase stays cleaner as it grows. 2. Debugging Data Pipelines When Spark jobs fail or SQL breaks in production, LLMs help translate ugly error logs into plain English. They can suggest troubleshooting steps or highlight what part of the pipeline to inspect next, helping you zero in on root causes faster. If you’re stuck on a recurring error, LLMs can propose code-level changes or optimizations you might have missed. 3. Documentation and Knowledge Sharing Turning notebooks, scripts, or undocumented DAGs into clear internal docs is much easier with LLMs. They can help structure your explanations, highlight the “why” behind key design choices, and make onboarding or handover notes quick to produce. Keeping platform wikis and technical documentation up to date becomes much less of a chore. 4. Data Modeling and Architecture Decisions When you’re designing schemas, deciding on partitioning, or picking between technologies (like Delta, Iceberg, or Hudi), LLMs can offer quick pros/cons, highlight trade-offs, and provide code samples. If you need to visualize a pipeline or architecture, LLMs can help you draft Mermaid or PlantUML diagrams for clearer communication with stakeholders. 5. Cross-Team Communication When collaborating with PMs, analytics, or infra teams, LLMs help you draft clear, focused updates, whether it’s a Slack message, an email, or a JIRA comment. They’re useful for summarizing complex issues, outlining next steps, or translating technical decisions into language that business partners understand. LLMs won’t replace data engineers, but they’re rapidly raising the bar for what you can deliver each week. Start by picking one recurring pain point in your workflow, then see how an LLM can speed it up. This is the new table stakes for staying sharp as a data engineer.

I work at Airbnb where I use LLMs to add features to huge codebases with custom libraries and APIs. LLMs aren't only for new projects. Here is my workflow: Treat the LLM like a new developer that needs to ramp up on the project 𝟭. 𝗦𝘁𝗮𝗿𝘁 𝘄𝗶𝘁𝗵 𝗮 𝘀𝗽𝗲𝗰𝗶𝗳𝗶𝗰 𝗲𝗻𝘁𝗿𝘆 𝗽𝗼𝗶𝗻𝘁 I ask my coding agent to start at the root directory and read a specific file path related to the feature or area I’m working on. From there, I have it trace the code and its dependencies so it can understand how the system fits together. 𝟮. 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗲 𝗶𝗻𝘁𝗲𝗿𝗻𝗮𝗹 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 I ask the agent to crawl through the codebase and draft documentation explaining how that feature or area works. It looks at imported libraries, API schemas, shared utilities, common patterns, endpoints, and related functions. Basically, it does what a human engineer would do when ramping up on an unfamiliar part of the codebase without much documentation. 𝟯. 𝗪𝗿𝗶𝘁𝗲 𝘁𝗵𝗲 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝘁𝗼 𝗱𝗶𝘀𝗸 I have the agent save the output into a docs/ folder. This can include written explanations, implementation notes, and Mermaid diagrams that show how components relate to each other. That documentation becomes reusable context for the rest of the task. 𝟰. 𝗥𝗲𝘃𝗶𝘀𝗲 𝘁𝗵𝗲 𝗱𝗼𝗰𝘀 𝗮𝘀 𝗻𝗲𝗲𝗱𝗲𝗱 The first version is rarely perfect, so I review and refine it. Once it’s accurate, I can ask the LLM to read those docs before implementing the feature. From that point on, it has much better context for the codebase. 𝟱. 𝗔𝗱𝗱 𝘁𝗵𝗶𝗿𝗱-𝗽𝗮𝗿𝘁𝘆 𝗹𝗶𝗯𝗿𝗮𝗿𝘆 𝗰𝗼𝗻𝘁𝗲𝘅𝘁 When I’m using an external library, I point the agent to the relevant external documentation pages. Then I ask it to summarize the important methods, APIs, return values, examples, and best practices into another file in the docs/ folder. This helps the agent understand not just our codebase, but also the tools it needs to use correctly. 𝟲. 𝗨𝘀𝗲 𝗰𝗵𝗮𝘁 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝘀 𝘄𝗵𝗲𝗻 𝗻𝗲𝘁𝘄𝗼𝗿𝗸 𝗮𝗰𝗰𝗲𝘀𝘀 𝗶𝘀 𝗹𝗶𝗺𝗶𝘁𝗲𝗱 If the coding agent doesn’t have network access, I do this step manually with ChatGPT, Gemini, or Claude. I drop in the relevant docs link and ask for a concise summary focused on the APIs and patterns I’ll need. For this kind of deep documentation and codebase-understanding work, I’ve found GPT-5 Pro, Gemini 3 Pro, and Opus 4.7 with extended thinking especially useful.

The enterprise LLMs wave is building up. 🌊 An insider at a large fintech gave me the play-by-play on their company's LLM launch. Here's what I learned: 🚀 Impact areas LLMs excel at customer support and generating code from English specifications. They're also applied to deduplicate data and, not surprisingly - explain code. 🧪 Testing rigor They use consistent benchmarks to evaluate different LLMs head-to-head across accuracy, cost, and speed. This enables data-driven selection of the optimal model for each use case. 📄 Data Licensing They are quite sophisticated at navigating legal agreements around licensing datasets to train language models. ☁️ Deployment models Both public API access and internal hosting have roles depending on the use case. Code generation favors internal models trained on company data. 🔎 Monitoring quality They track document usage to ensure high LLM quality on frequently accessed items. They want to make sure the top accessed documents are extremely high quality. 🎯 Defining success 95%+ accuracy on real-world problems is the threshold for adoption. Anything less is not defendable to the CEO and the Board. 💡 Consultative AI Expertise They prefer consultative guidance from AI experts over vendor-provided tools when measuring AI performance. In a word, pragmatism is the way to go! Does this approach reflect your own evaluation processes so far? 📊

A second-order effect of AI tools - businesses are increasingly looking for OaaS (Outcome as a Service) rather than SaaS (Software as a service). For the last few decades - SaaS companies have pitched their products as productivity enhancers for average employees. The business logic was that your employee who earns $50,000/year is now 25% more productive. You can purchase our software for $25,000 annually, hire 8 employees instead of 10, and save $75,000. But LLMs can give highly-productive employees superpowers. This means the employee who is earning $200,000/year is now 100% more productive. But in the real world - salaries and workloads aren’t that flexible. Employers can’t double employees’ scope of work or reduce salaries by 50% overnight.    The solution that companies are looking for is ‘buying work, not just software’. You have an AI tool for auditing financial documents? Bundle it with a qualified human who is in the loop. An AI code generator? Add a developer who is qualified to review and sign off on it. The latter isn’t even hypothetical - Google just announced last week that 25% of its code is AI-generated - with employees only reviewing and accepting it. My guess is that you’ll probably see a new kind of AI-first marketplace opening up on the lines of Upwork or Fiverr. With highly-qualified experts who leverage LLMs to produce fast, cheap high-quality (and high-margin) output.

Is code still intellectual property in a world where AI can generate it on demand? The rise of LLMs is transforming the foundation of software development, challenging traditional notions of intellectual property. Historically, IP was straightforward to identify: it was the code itself. Companies invested months or years building proprietary algorithms and implementations that represented their crystallized knowledge of how to solve complex problems. Today, LLMs can generate sophisticated, production-quality code in minutes rather than months. While this code still has inherent value in its utility, its nature as intellectual property has fundamentally changed. When multiple developers can prompt an LLM to generate similar solutions, the code itself no longer represents unique intellectual capital. This capability not only democratizes software development but raises a profound question: If code can be generated so easily, where does the real value in software development now lie? The answer is that intellectual property in the LLM era isn't in the code itself, but in the empirical knowledge of what actually works in practice. While LLMs can generate theoretically perfect code, they can't capture the practical insights that only come from deploying solutions in the real world. Consider a complex integration between enterprise systems - an LLM might generate syntactically correct code, but only practical experience reveals which approaches handle edge cases, maintain performance at scale, and remain maintainable. The true IP becomes this accumulated knowledge of real-world implementation challenges. This shift fundamentally changes how we build software businesses. Success is no longer about accumulating proprietary code, but about systematizing practical knowledge of making LLM-generated solutions work reliably at scale. When an LLM can generate ten different valid approaches to a problem, the competitive advantage lies in knowing which approach will actually succeed in production, how to implement it robustly, and how to maintain it efficiently. This knowledge, unlike code, can't be easily replicated or generated. We're entering an era where the winners won't be those who write the best code—that's becoming commoditized. Instead, they'll be those who most effectively accumulate and apply practical knowledge about making LLM-generated solutions work in production. This represents a profound shift: while LLMs can generate multiple, seemingly valid solutions to any problem, their non-deterministic nature means each generation might produce different code. This uncertainty actually increases the value of implementation knowledge - success requires understanding not just one solution, but the patterns and principles that work reliably across many possible implementations. The true intellectual property is the battle-tested knowledge of how to navigate from AI-generated possibilities to production-ready solutions.