Jonathan Foley

Foundational models, or large language models (LLMs), train at the scale of the entire internet to acquire the vocabulary, syntax, and narrative structure of human language. Fine-tuning involves using a smaller, domain-specific dataset to grant additional specificity to a pre-existing foundational model that has already acquired patterns and features from massive underlying training datasets. By utilizing the existing knowledge embedded in a pre-trained language model, we unlock high-performance language understanding, but fine-tuning helps to get the right answer - not just an answer that sounds right - with a minimum of hallucinations or incorrect info. In healthcare, out-of-the-box, foundational models are often too generic to sufficiently answer the precision of a healthcare query. Foundational language models are trained on a massive scale of data, which includes jokes, incorrect information, misunderstandings, and misinformation, along with the sum total of human knowledge. The problem is, if you ask a large language model to give you factually correct information when it has been trained on BOTH accurate and inaccurate information - you’re playing a bit of a dangerous game. Enter fine-tuning language models. In the old days, we used to build a single model for a single task - for example, classifying procedures into evidence-based or not as laid out by clinical practice guidelines, for patients with particular disease markers. Each disease would have its own model, which took a long time to build and didn’t scale particularly well, and we struggled with the sparsity and inherent high-dimensionality of healthcare data. By fine-tuning LLMs, however, we're able to specialize those foundational models to achieve greater domain specificity and specialization, leverage the unique predictive and planning capabilities of LLMs, and cut down on the incorrect information that such models inevitably produce, while still preserving the underlying language prowess of the LLM. If improved accuracy and specificity wasn't enough of a selling point, there's another notable upside to fine-tuning foundational models for use in a clinical context. In terms of computational efficiency, using LLMs out of the box on specific tasks can require massive compute (and spend) that amounts to computational overkill for the problem. Some LLMs charge per query, some by compute load or time - either way, as a business running LLMs in a production environment, minimizing the number of LLM calls means optimizing your compute expenditures, too. When LLMs fail at scale in healthcare, it's usually along the fault line of failure to specify adequately. Fine-tuning large language models along domain-specific lines is one way to effectively productize genAI for healthcare use cases. #genAI #LLM #ai #healthcareAI #healthtech #largelanguagemodels #generativeAI #chatGPT

61

2 Comments

I wanted to share some insights from a recent paper from Amazon Science in which authors tackled an important challenge in language modeling: minimizing sequence truncation. In language models, truncating input sequences might lead to a loss of crucial information, especially when the models are dealing with long-context tasks such as document summarization or story generation. The authors introduce a clever solution by leveraging a heuristic from the Best Fit Bin Packing problem, an NP-hard problem, to reorder input documents and reduce truncation. The proposed approach helps retain more information compared to other strategies by optimizing how sequences are packed into the model's context window. Authors show that the bin packing solution had biggest performance gain reading comprehension (+4.7%), natural-language inference (+9.3%), context following (+16.8%), and program synthesis. They also found that best-fit packing helped prevent closed-domain hallucination. Authors claim that while the experiments primarily focused on LLM pretraining, best-fit packing is broadly applicable to fine tuning as well. https://lnkd.in/gWquw3ud

28

3 Comments

Sharing some of our research at M42 Health on evaluating Large Language Models (LLMs) 📊 🤖 Some quick key takeaways: 1️⃣ Evaluating LLMs is far from being trivial, with challenges present even in seemingly simple tasks such as answering multiple-choice questions (MCQs). 2️⃣ We show that different evaluation methodologies can lead to discrepancies between 5 and 26% in reported accuracy for the same model and task/dataset! 3️⃣ This research highlights the need for robust and standardized evaluation frameworks, and greater transparency in the reporting of both performance metrics and evaluation methodologies used to allow better cross-model and cross-study comparisons. 4️⃣ Finally, it shows the important role of evaluation frameworks in the standardization of assessment of LLMs, such as EleutherAI's harness framework (used by some leaderboards Hugging Face), Stanford University's HELM, OpenCompass, among others. The paper also offers a short analysis of popular MCQ benchmarks and some of these methodologies. Read more at https://lnkd.in/dnveBx4c Joint work with Clément Christophe, PhD, Praveenkumar Kanithi, Tathagata Raha, Prateek Munjal and Shadab Khan #AI #MachineLearning #NLP #LLM

79

2 Comments

Open-source AI for evaluating language models Researchers from KAIST AI, LG AI Research, CMU, MIT, Allen Institute for AI, and UIC have introduced Prometheus 2, a groundbreaking open-source evaluator model that assesses language model outputs with accuracy rivaling expensive proprietary solutions like GPT-4. A few key highlights: Prometheus 2 combines models trained on direct assessment and pairwise ranking, enabling it to excel at both common evaluation formats. It leverages the new Preference Collection dataset with 1,000 evaluation criteria to boost performance. Prometheus 2 achieved the highest correlation with human judgments and proprietary models across 8 benchmarks, in some cases reaching 0.90 Pearson correlation and 85%+ accuracy. This helps narrow the gap between open-source and closed-source AI evaluation without sacrificing transparency and control As an AI and Data Strategist, I like that this will make high-quality AI evaluation more accessible and scalable for researchers and practitioners. Some potential strategic impacts: Accelerating AI/ML system development by enabling rapid, reliable evaluation of generated language outputs at lower cost Establishing clearer standards and benchmarks for assessing language model performance as open alternatives mature #ai #machinelearning #nlp #opensource #research #aistrategy Paper - https://lnkd.in/gsnGDwJd

1

Last weekend, I read "Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought" by Lee et al, 2024. In this paper propose to use small (<1B params) language model (SLM) to provide a sequence of rationale steps to a LLM to predict the output of a task. The authors argue that this is an efficient way to use LLMs to solve a task by only training the SLM that provides the rationale to the LLM. To me, this approach is interesting because they use reinforcement learning by assigning reward signals for a SLM from a rationale-oriented and task-oriented perspectives to improve the outputs of a LLM. Enjoy! Paper: https://lnkd.in/gcQwW-ty

3

When *open* isn't *open source* 🙋🏼‍♂️👀 Reflecting on the open-source landscape for Large Language Models, I explore the essentials: permissive licensing, transparency in training data and model architecture & more #LLM #LLMs #AI #GenerativeAI #opensource https://lnkd.in/gMxbN4Nb

35

2 Comments

Explainable AI - What actually is it? Explainable AI is reentering the discourse...after a rather quiet period (because let's face it - it is really hard for LLMs). I wanted to bring up several important questions to address when deciding to discuss or work on this topic. There is room for all of the answers, but DO have a clear answer before you jump into this space. 1. What does "Explainable" actually mean in your context? Not super obvious. Are you trying to explain the exact mathematical/programmatic set of steps that went on between the input and the output? This works well for a Logistic Regression or a Rule-base model, but not so much for a multi-billion parameter model. Are you trying to approximate which features of the input had largest effect on the output? Can be applied to large models, but many explanation methods will have a degree of error. Are you trying to explain the underlying "human" process (captured in the training data) that connected the inputs and the outputs? Arguably more useful in certain contexts, but may not connect to the actual model internals. 2. Why are you creating explanations? Many different worthwhile goals here. They will help answer question 1, and the general path you take. A few possibilities: - Building Trust: Proving to a human that a model is doing something meaningful and accurate. - Evaluating for Bias and Unfairness: Proving to a human that a model is not relying to "biases and stereotypes" to make decisions. - Progressing Research: Fundamentally how a model works can help build better models. 3. Who are you building explanations for? Explanations that are helpful for an ML Engineers are not the same ones that are helpful for a doctor. An ML Engineer may want a higher-fidelity approach that "highlights" the weights in the model; a doctor may want to see several reference MRI scans from the training data that led to a diagnosis but not actually care about the underlying neural network. ⁉ What other fundamental questions did I miss? PS: I chose not to address the notions of "interpretability" vs "explainability" here because that would get us way past the character limit.

7

A recent study (Nori, et. al) shows that a well prompted foundation model, like GPT-4, can achieve results comparable to fine-tuned models such as BioGPT or Med-PaLM. This is significant because effective prompts can be created in a much more affordable manner than fine-tuning an LLM. Check out the study here: https://lnkd.in/eCsitTA8

22

Function calling is a powerful way to extend the capabilities of LLMs and AI agents by letting them use external tools. Our new short course Function calling and Data Extraction with LLMs, created with Nexusflow and taught by Jiantao Jiao and Venkat Srinivasan, demonstrates how to prompt LLMs to form calls to external functions. You'll work with NexusRavenV2-13B, a 13B parameter open-source model that excels in function calling tasks while still being small enough to host locally. Learn to use function calling to extract structured data from unstructured text and access web APIs, and build an end-to-end application that processes customer service transcripts. You'll learn how to build LLM-powered applications that can analyze feedback, automate data entry, and enhance search. Please get started here: https://lnkd.in/g9FpiCuH

5,045

114 Comments

The Rise of Agentic-Architectures in Generative AI: Old wine in a new bottle?      Tldr; As generative AI evolves, developers are increasingly recognizing the need for sophisticated orchestration of Large Language Models, Retrieval Augmented Generation, and multi-agent architectures to handle complex queries, prompting us to better leverage established multi-agent systems and AI planning techniques in the field of Computer Science to build more capable and efficient agents.     ——— As developers gain a deeper understanding of the challenges in building and deploying generative AI applications such as chatbots and copilots with Large Language Models (LLMs), Retrieval Augmented Generation (RAGs), and agents, they are recognizing the limitations of solely relying on prompt engineering and basic agent architectures. To address complex, real-world challenges, sophisticated orchestration of prompts, LLMs, and specific task execution tools is required.      For example, a generative AI solution with simple and single agent architecture even with a powerful LLM is unlikely to address complex multi-part queries such as ‘compare the revenue of company XYZ from Q1 through Q4 of FY2024 and provide an analytical commentary on the key contributing factors that led to the changes in revenues during this time’.  These kinds of use cases are driving increasing interest towards multi-agent architectures capable of complex task decomposition and more.        So, naturally, these days, we are increasingly noticing discussions, and papers about agentic architectures for generative AI solutions. Interestingly, many of the design patterns that are gaining traction in agent architectures are reminiscent of earlier research in Multi-agent Systems (MAS) and AI planning in the field of Computer science. Old wine in a new bottle? Well, time will tell. In any case, some noteworthy agent patterns from previous works include:     * Layered Architectures: Enable task decomposition through hierarchical agent structuring.   * Blackboard Architectures: Facilitate independent agent work with a communal 'blackboard' for sharing results, ideal for diverse expertise collaboration.   * Belief-Desire-Intention (BDI) Models: Support planning and decision-making with agents driven by their beliefs, desires, and intentions.   * Contract Net Protocol (CNP): Uses a decentralized protocol for task allocation where agents bid on tasks based on their capabilities.   * Market-Based Architectures: Inspired by economic models, these architectures use auctions and currency systems for efficient resource allocation.   * Hybrid Architectures: Combine various frameworks to leverage the strengths of different systems, creating robust and flexible solutions.     As we dig deeper into the world of agentic architectures for generative AI, the wisdom from decades of research in MAS, and AI planning domains seem increasingly relevant. So, let's build on prior work and not reinvent the wheel.  #genAI

216

23 Comments

The reasons why AI-powered search stinks at sourcing are both technical and simple, Shah explained. The technical explanation involves something called retrieval-augmented generation (RAG), which works a bit like a professor recruiting research assistants to go find out more information about a specific topic when the professor’s personal library isn’t enough. RAG does solve a couple of problems with how the current generation of large language models generate content, including the frequency of hallucinations, but it also creates a new problem: It can’t distinguish good sources from bad. In its current state, AI lacks good judgment......

2

Just read Farquhar et al's Nature paper on detecting hallucinations in large language models using semantic entropy. Interesting result on quantifying uncertainty in LLMs. When you ask an LLM a question, its answers may look correct, but be made up - a confabulation. For example, the LLM may answer incorrectly that the STARD10 protein is a "negative regulator of the mTOR pathway" instead of its true role as a lipid transfer protein. When an answer is confabulated, multiple subsequent generations will have different meanings. For example, a second answer of the LLM may incorrectly say that STARD10 is a "negative regulator of the meiotic recombination process". The variability in meaning can be quantified using semantic entropy, which measures variability over multiple generated answers. Higher entropy suggests greater variability and potential confabulation. The LLM can then refuse questions with high semantic entropy. To calculate semantic entropy, you generate 5-10 answers. You then cluster the answers based on semantic meaning using bi-directional entailment. Two answers are part of the same cluster if they entail - or imply - each other (e.g. "Joe buys a red apple" implies "A red apple was bought by Joe" and vice-versa). The entailment is calculated using another, possibly less powerful LLM, so it's an interesting applications of using language models to correct other language models. After clustering, the probabilities of each cluster are calculated from the output probabilities of the LLM, and then the resulting entropy. Thresholding the entropy can classify a question as likely to produce confabulations with an AUROC of around 0.75. Link to paper below.

63

2 Comments

We recently had Jon-Saad Falcon from Stanford AI Group give a talk at SambaNova Systems. He talked about his research ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems. You can check out the talk at - https://lnkd.in/eXkBTb-j RAG systems are really hard to evaluate. Lot of hand holding required with annotated data or manual inspection. Excited to see this research and its outcome to help alleviate this problem. Key to this idea is a composite system using LLM as a judge. LLM judges are trained using synthetic data on domain queries/answer pairs. Its exciting to see how our customers use SambaNova Systems Platform to enable these workflows efficiently. Exceptional speed allows for fast synthetic data generation! And ability to host multiple models on a single system with extremely fast model switching allows for efficient low cost deployment. Try out 405B running at 120+ tokens/second at https://fast.snova.ai/ Want to enable your own such workflows by hosting fine-tuned checkpoints? Sign up here - https://lnkd.in/ecAsSDRH

59

I recently read “Towards Monosemanticity: Decomposing Language Models with Dictionary Learning” by Bricken et al 2023. Published at transformer-circuits.pub In this paper the authors discuss the use of autoencoders to extract interpretable features from a one-layer transformer model. The approach aims to improve the understanding and transparency of artificial neural networks. To do this they use the concept of dictionary learning which is a method used to decompose the activations of the transformer model into a set of basis features that are sparse and interpretable. This involves training sparse autoencoders to identify a dictionary of monosemantic features (or features with a single meaning). These monosemantic features in single units (artificial neurons) reminds me of the “grandmother neuron” theory in neuroscience which proposes that specific neurons respond to specific highly complex stimuli. However, in real brains, it is widely believed that complex representations are encoded by the collective activity of groups of neurons (and not monosemantic). But going back to the paper, I really like the visualization tools and illustration to make such complex networks more transparent (as opposed to black boxes). Paper: https://lnkd.in/gR87pJcs

3

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

🎉 Excited to share our paper, "IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation" has been accepted to ECCV 2024! 💃 IDOL brings your favorite characters to life in 2.5D, simultaneously generating high-fidelity animated videos and depth maps. 🔑 Key contributions - Unified dual-modal U-Net for joint video-depth generation - Motion consistency loss for precise video-depth alignment - Cross-attention map consistency loss for enhanced spatial coherence Huge thanks to my incredible intern mentors Kevin (Ke-Yun) Lin, Linjie Li, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Zicheng Liu, and Lijuan Wang! 👉 Project page: https://lnkd.in/ehPdK3DY 👉 Paper: https://lnkd.in/e2cfhcjJ 👉 Code (coming soon): https://lnkd.in/ey4vcC9m

49

This course/conference is one of the most interesting bits of learning I've done in years. There is a growing spectrum of use cases with Generative AI across a wide range of possible investment. It was absolutely fascinating to see how the sausage is made with real-life application of these models. There is a lot more tooling and frameworks than I realized - we aren't at "xgboost in a notebook on your machine" levels of ease yet, but it's actually really close. I wasn't prepared for how much GenAI is used to create and curate data for GenAI. And I'm now on the marketing lists of so many different cloud GPU providers who are fighting over who gets to add another few billion dollars to Jensen Huang's net worth. This ecosystem is exploding. And unlike crypto, there's something real at the center of it.

8