Generative AI is here: What are we going to do about it?

Arfon Smith / 1 October 2024 Cross-VISS Convening Generative AI
is here What are we going to do about it?

Cross-VISS Convening Generative AI is here: What are we going
to do about it? Plan for today 1. Getting aligned     2. What we’re learning about building with LLMs    3. Making preparations for the future Current capabilities, what’s readily possible, where some challenges lie, some worked examples. Some common architectures, techniques, and product principles. Getting opinionated about how to use these technologies, attempting to predict the future.

Software as a creditable research activity

Cross-VISS Convening Zenodo & fi gshare archives https://gh.io/doi

Cross-VISS Convening Zenodo DOIs for software releases https://gh.io/doi

Cross-VISS Convening Citation Files https://gh.io/cite

Cross-VISS Convening https://gh.io/orcid ORCID

Cross-VISS Convening ORCID https://gh.io/orcid

to do about it? Generative AI is here

to do about it? Some working assumptions Based on the last ~2 years actively building with them The people around you are likely using it There is a “there” there “Moore’s law for LLMs” will continue to hold People are going try all sorts of crazy sh*t As builders of technology, we should have opinions Whether it’s ChatGPT or GitHub Copilot, these are technologies people are using. Generative AI can be genuinely useful when applied to the right problems. Models will likely become more capable, costs will reduce, more will be possible for less. Just check your favourite tech news site. And I’m sharing mine with you today.

to do about it? Quick survey of the room Using generative AI in their daily work? Building a system that incorporates LLMs? Exploring capabilities as part of their work? Living with the consequences of LLMs in their work? Building their own base model? e.g., GitHub Copilot (or other code tool), ChatGPT, Claude, something else. Building a net-new piece of infrastructure or building something new? e.g., Evaluating models for existing or future workloads? e.g., A collaborator using generative AI tools. Advanced mode… With a show of hands…

to do about it? Core capabilities and challenges What are we learning?

to do about it? What are they capable of? Natural language processing tasks Synthesizing information Personalization (ELI5, ELIM, ELIDKCVW*) Fine tuning for domain-speci fi c tasks or behaviours Exploring topics, generating ideas. Text classi fi cation, sentiment analysis, named entity recognition, intent detection. Especially when combined with techniques like Retrieval Augmented Generation (RAG) Customized responses based on individual preferences/background/knowledge. Conversational agents, code generation, tool calling. With some caveats, can be excellent tools for brainstorming and learning. * Explain Like I Don’t Know C Very Well

to do about it? What are some challenges? Hallucinations Biases, safety, security Inconsistency, non-determinism, and evaluation Structured outputs Unjusti fi ed certainty in outputs They are always hallucinating, it’s just sometimes they are useful. Re fl ecting, and sometimes amplifying biases present in training data. Di ff erent answers for the same questions due to random sampling and model state. Although many models have now been trained for this speci fi cally. Generate detailed responses without any sense of reality.

to do about it? How are people using LLMs?

to do about it? Hacker News results for “AI” 27,000+

to do about it? Hacker News results for “LLM” 3,500+

to do about it? GitHub repositories mentioning LLMs 1M+ https://github.com/Shubhamsaboo/awesome-llm-apps

to do about it? Some worked examples GitHub Copilot (IDE, Chat, and GitHub.com)

to do about it? A worked example: Copilot Code Completions

to do about it? APA and BibTeX citation strings ruby-c f https://github.com/citation-file-format/ruby-cff

to do about it? Completions response = openai.Completion.create( engine="davinci-codex", prompt="module CFF\n # Generates", temperature=0, max_tokens=150, top_p=1, frequency_penalty=0, presence_penalty=0, stop=["#", "```", "\n\n"] ) print(response.choices[0].text)

Life of a completion Open editor tabs Editor session data
Prompt library Contextual fi lter model GPT model (e.g., Codex) Prompt Generated completion(s) Local IDE Server Gather context.

Prompt library Contextual fi lter model GPT model (e.g., Codex) Prompt Generated completion(s) Local IDE Server Assemble prompt.

Prompt library Contextual fi lter model GPT model (e.g., Codex) Prompt Generated completion(s) Local IDE Server Generate completions.

Prompt library Contextual fi lter model GPT model (e.g., Codex) Prompt Generated completion(s) Local IDE Server All completed in ~300ms 💨

to do about it? You are an AI programming assistant. When provided with code snippets, analyze them and provide helpful suggestions or corrections. Complete the following code snippet below for the user. Related code: # Filename: apa_formatter.rb module CFF # Generates an APALIKE citation string class ApaFormatter < Formatter # :nodoc: def self.format(model:, preferred_citation: true) # rubocop:disable Metrics/ AbcSize model = select_and_check_model(model, preferred_citation) return if model.nil? ... # Current filename: mla_formatter.rb module CFF\n # Generates

https://github.blog/ai-and-ml/llms/the-architecture-of-todays-llm-applications/

to do about it? Copilot Chat

to do about it? Quick aside about LLM behavior

to do about it? Models require tuning to be useful INPUT/PROMPT What was the fi rst state in the USA? What was the second state in the USA?

to do about it? Models require tuning to be useful INPUT/PROMPT What was the fi rst state in the USA? What was the second state in the USA? What was the third state in the USA? What was the fourth state in the USA?

to do about it? Fine tuned with (lots of) examples What does attr_accessor mean in ruby? attr_accessor in Ruby is a method that automatically creates getter and setter methods for instance variables. Why can't you mix types in a list in Python? I apologize, but there's actually a misunderstanding in your question. In Python, you can indeed mix types… How do I de fi ne a variable in Fortran 77? Fortran 77 has a di ff erent approach to variable declaration compared to modern Fortran versions… What are threads in programming? Threads in programming are lightweight units of execution within a process. They allow a program to… What was the fi rst state in the USA? The fi rst state to ratify the United States Constitution and o ff i cially join the Union was Delaware. (And for a speci fi c behavior)

to do about it? Copilot Chat

to do about it? Copilot Chat "messages": [ { "role": "system", "content": "You are an AI programming assistant. When provided with code snippets, analyze them and provide helpful suggestions or corrections." }, { "role": "user", "content": "What other publication types should I consider supporting here?" }, { "role": "user", "content": "I have the following Ruby code snippet:\n\ndef self.publication_data_from_model(model) # rubocop:disable Metrics/AbcSize, Metrics/ CyclomaticComplexity, Metrics/MethodLength\\n case model.type\\n when ‘article’\n\n... }, { "role": “assistant", "content": "..."

to do about it? Tools

to do about it? Tools An LLM fi ne tuned for function calling Introduced initially by OpenAI Allows for LLMs to be connected with existing tools Models themselves *do not* actually invoke tools Used to support a wide range of scenarios Detect when a function to be called, and return that function (as JSON) with arguments. GPT 3.5 and GPT 4 variants, now supported by many (non-OpenAI) models. Retrieve additional context (as required) e.g., based on the user’s input. Return structured outputs that can then be used in between trips to model. Code interpreter for ChatGPT, retrieval of context for chat interfaces, sentiment analysis. (Also known as function calling)

Tool calling chat interface

“What tools do you have available”

Where is the GetAuth method implemented? User input Copilot Agent
GetSymbol CodeSearch GetDiff GetIssue GetFile SearchWeb Tools 1. (LLM) Select tool(s) 2. Execute tools 3. (LLM) Generate response Symbol Search CodeIndex Git APIs GitHub API Git APIs Bing API Tool calling fl ow APIs / Services

“How do components manage state in React?”

"messages": [ {"role": "system", "content": "You are an AI programming
assistant. When provided with code snippets, analyze them and provide helpful suggestions or corrections."}, {"role": "user", "content": "How do components manage state in React?"} ], "tools": [ { "type": "function", "function": { "name": "search_bing", "description": "Search the web using Bing and return the top results", "parameters": { "type": "object", "properties": { "query": { "type": "string", "description": "The search query to send to Bing" }, "num_results": { "type": "integer", "description": "The number of search results to return", "default": 3 } }, "required": ["query"] }

"messages": [ {"role": "user", "content": "How do components manage state
in React?"}, {"role": "assistant", "content": "To provide you with the most up-to-date and comprehensive information about state management in React components, I'll need to search for the latest resources. Let me do that for you."}, {"role": "assistant", "function_call": { "name": "search_bing", "arguments": "{\"query\": \"How do components manage state in React?\", \"num_results\": 3}" }}, {"role": "function", "name": "search_bing", "content": "1. React components can manage state using the useState hook. This allows functional components to have state variables. \n2. Class components can manage state using this.state and this.setState().\n3. For more complex state management, libraries like Redux or React Context API can be used.\n4. The useReducer hook is another option for managing more complex state logic within a component.\n5. React also provides the useEffect hook for managing side effects related to state changes."}, {"role": "assistant", "content": "Based on the latest information, there are several ways components manage state in React: 1. useState Hook: For functional components, the useState hook is the primary way to manage state. It allows you to declare state variables and provides a function to update them.

to do about it? What are people building upon? Platforms, models, tools, frameworks…

to do about it? Technologies to build upon OpenAI, Azure AI, Hugging Face, AWS Bedrock Open and closed models, model inference endpoints, fi ne tuning, data retrieval services for quickly building out common architectures (e.g., RAG), basic safety tooling for reducing end-user risk. PLATFORMS LIBRARIES LangChain, LlamaIndex, ONYX, Transformers Tools for evaluating models locally. Frameworks to incorporate LLMs reliably by creating interoperable components. MODELS GPTs, Claude, PaLM, LLaMA… And a lot, lot more. Multi-modal, fi ne-tuned for di ff erent behaviours (completions, conversation, function calling), ‘distilled’ models. Could do a talk devoted to this subject alone.

to do about it? Topics to learn more about Prompt engineering  Understanding how to craft e ff ective prompts is essential. Fastest way to learn is in some kind of playground environment where you can develop an intuition for how to structure prompts to elicit desired responses.        Few-shot learning and fi ne-tuning  LLMs can do a lot ‘out of the box’, few-shot learning (in the prompt) can be very e ff ective at guiding behavior. In some specialized cases, fi ne-tuning of a model may be necessary.        Evaluation and safety  Robust evaluation is critical for assessing an LLM-based system's performance and reliability. Lots of active research going on here, and di ff erent techniques for scoring outputs. Understanding potential risks such as biased outputs, hallucinations (generating false information), and potential misuse is essential for building safe and responsible AI systems.

to do about it? Key takeaways LLMs are weird Building an MVP is easy, building a product is hard Speak like a human (when prompting them) Build an evaluation framework/approach early Leverage ‘paved paths’ Prompting is a real skill, and it takes time to develop intuition. Not new, but worth repeating a million times over. They’ve been trained on lots and lots of human writing. Work out what success metrics are and how to measure them. Lots of hosted platforms, open source tools, and evaluation frameworks to use.

to do about it? Product principles for builders Derived from fi rst hand experiences and lots of conversations with experts

Cross-VISS Convening No decisi on s

to do about it? No decisions Copilot in the ‘right seat’ Products that assist rather than control LLMs make mistakes – building assistive experiences signi fi cantly safer than ones that take action directly. Lots of opportunities in this mode e.g., Accelerating context, presenting a range of options to the user to make a decision on. Founding principle for GitHub’s Copilot Code completion, Chat experiences, GitHub Workspace are all designed to assist human.

to do about it?

Cross-VISS Convening Design f or fail ur e

to do about it? Design for failure What is the cost of getting it wrong? Are there opportunities for review? If an error occurs, is the user likely to spot it? What is the cost to the user if they don't? What is the experience ‘tax’ of failure? If detected, how easy is it to dismiss an error and move on? Measuring good Can you measure a good (and bad) experience?

Cross-VISS Convening

to do about it? An inherent characteristic of LLMs Hallucinations are unavoidable Mild hallucinations are very common Lots of techniques and tools exist for mitigating them Did I say they are unavoidable? They are *always* hallucinating, it’s just sometimes those hallucinations are useful. But can be reduced through a variety of techniques including tuning, prompting, context. Design for roughly 50% of the time. Severe ones less common, especially with grounding. Active area of research and development. Responsible development means only building systems where hallucinations aren’t dealbreakers. AI/LLM hallucinations A quick aside about…

Cross-VISS Convening Gr ou nded in reality

to do about it? Grounded in reality Optimizing the output of large language models Giving models the ability to gather context This context is used in the prompt to steer behavior. Tool calling is an example, as is RAG… Retrieval-Augmented Generation (RAG) …is probably the best ‘bang for buck’ here. Lots of emergent architectures. Fine tuning for very speci fi c cases e.g., for a particular task, but *not* for embedding signi fi cant new knowledge. Should rule out prompting approaches fi rst.

Cross-VISS Convening Retrieval-Augmented Generation 🧑💻 LLM Find related information Prompt
+ user query Prompt + user query + retrieved context Blackbird, Git/Spokes, GitHub API/graph

to do about it? Retrieval Augmented Generation Lots of di ff erent ways to do the ‘retrieval’ Essentially giving the model relevant input context Combined with links/citations can increase utility Allows base models to ‘work’ for specialised areas Typically much cheaper than other optimisations Keyword search, vector search, other ‘similarity-type’ searches. Steers the model outputs towards better/more relevant answers. Allow users to follow relevant links, decide which of them are most relevant. For example if the topic being discussed isn’t ‘knowable’ from public information. e.g., fi ne-tuned (or completely custom) models.

Cross-VISS Convening Invite on ly

to do about it? Invite only It looks like you're writing software. Would you like help? LLMs can power remarkable experiences I am an employee of a wholly-owned subsidiary of Microsoft – the company that brought Clippy into the world. Only surprise if you can delight Remember, small hallucinations are very common.

Cross-VISS Convening Explain like it’s me

to do about it? Explain like it’s me Leveraging the LLM’s ability to personalise responses An inherent quality of LLMs Models are very good at rephrasing information based on rules/ instructions. Instructions in prompt or conversation Customization can happen ‘dynamically’ (ELI5), or as a set of ‘custom instructions’ that are passed to the model each time. Origin of (some) excitement about learning e.g., an AI assistant that can reframe/rephrase based on how a conversation is going.

Cross-VISS Convening No decisi on s • Design f or
fail ur e • Gr ou nded in reality • Invite on ly • Explain like it’s me

to do about it? Making predictions What might the future look like for research software engineering?

to do about it? Adoption and perception of AI tools Signi fi cant adoption in many fi elds 76% are using or are planning to use AI tools 81% cite increased productivity as biggest bene fi t Trust in output/accuracy of tools mixed 70% of developers do not perceive AI as a job threat 100M+ ChatGPT users, 1M+ GitHub Copilot users. Up from 70% last year. Novice developers cite accelerating learning as biggest bene fi t. 43% feel positive, 31% skeptical. Learners trust more than experienced developers. Those who *do* is marginally higher for learners (15% vs 12%). Stack Overflow

to do about it? Study: “The New Developer” Signi fi cant fraction of developers anxious about future Competitive engineers who identify with “innate brilliance” are most likely to cite an AI skill threat. Most developers are planning to upskill with AI 74% reported being very or extremely likely to seek ways to practice and apply new AI skills in software development. A strong learning culture can mitigate AI skill threat Where learning is encouraged, organizations can create an environment that's more resilient to the challenges posed by the transition to AI-assisted coding, while also boosting productivity and team e ff ectiveness. https://www.pluralsight.com/resource-center/guides/new-developer-whitepaper

to do about it? The (typical) role of the research software engineer Bridge the gap between research and software Facilitating collaboration, incorporating modern engineering practices, and advocating for best practices and technologies that can improve research (Chue Hong, 2020). Build technical capacity in their research groups Develop computational skills of wider group (PhDs, postdocs, PIs) through training and mentoring (Hettrick, 2018). Developing and supporting technical infrastructure Development and maintenance of research software, building computational models, data management infrastructure, computational research work fl ows (Crouch et al., 2019).

to do about it? Making predictions There will be more code… AI tools will enable more code to be created by more people (Research) software engineers will be able to code faster, but a lower barrier to entry will enable more people to participate in the software creation and code authoring process. CHat Oriented Programming (CHOP*) – “coding via iterative prompt re fi nement” Problem owners will be able to get closer to a solution independently – If you can provide a detailed speci fi cation to a solution, you might be able to get an MVP built without the help of a software engineer. Entering an era of maximal productivity for ex-coding PIs? If you know something about software engineering already then the currently available tools can be deployed to signi fi cant e ff ect already. * Sourcegraph: https://sourcegraph.com/blog/the-death-of-the-junior-developer

to do about it? Making predictions People will shoot themselves in the foot

to do about it? Making predictions Potential near-term challenges for RSEs AI tools may devalue coding expertise in the academy As more people rely upon AI generated code, RSEs' specialized skills might be undervalued, potentially reducing their in fl uence in research teams and projects. AI-generated code may compromise software quality Widespread AI-generated code production may diminish focus on best practices and long-term maintainability, challenging RSEs' role in ensuring high-quality research software. Shifting skill demands/expectations Those who don't adapt to incorporate AI tools into their work fl ow and develop new skills around AI-assisted coding might fi nd their traditional roles less relevant.

to do about it? Making predictions Potential near-term opportunities for RSEs Spending more time on complex (non-code) tasks RSEs can focus on solving intricate applied technology problems e.g., developing sophisticated computational models. Guide AI tool adoption in research teams RSEs can become essential personnel in teaching responsible* and e ff ective AI tool use, e.g., prompt engineering, leading conversations about adoption, thereby strengthening their role as technical mentors. Increased focus on architecture and integration RSEs shift focus to high-level software architecture and system design, leading to more more robust, scalable, and interoperable research software ecosystems for all. * Microsoft RAI standard: https://www.microsoft.com/en-us/ai/principles-and-approach

to do about it? Making predictions My recommendations Reviewing and testing code will be more important than ever If code has come from a mixture of humans and AI coding assistants, and a mixture of creators, the best practices around software review and testing become more important than even. If your team isn’t doing it, then start now. Embrace the opportunity, and lead from the front Decide as a team how you want AI to be leveraged in your team. Be the person that drives these conversations about thoughtful adoption of AI tooling into your daily work. Focus on levels of abstraction that encourage useful contributions Concepts like separation of concerns, loose coupling, and clear interfaces provide 'guard rails' for both AI- supported contributions. Lots of ideas to potentially leverage in the computational work fl ows ecosystem here.

Cross-VISS Convening Thanks! [email protected]

Generative AI is here: What are we going to do ...

Generative AI is here: What are we going to do about it?

More Decks by Arfon Smith

Other Decks in Programming

Featured

Transcript